Pith. sign in

REVIEW 3 major objections 6 minor 29 references

This paper argues that adapting a multilingual model to a new language and task need not require a joint fine-tuning run: separately trained language and task updates can be recombined in weight space, and a new cross-axis TIES rule beats t

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 03:09 UTC pith:UMDOCDUD

load-bearing objection A clean, honest empirical paper with a real (if modest) new twist on TIES — the summarisation headline holds up but rests on a 3-vs-1 seed comparison the authors openly flag. the 3 major comments →

arxiv 2607.13967 v1 pith:UMDOCDUD submitted 2026-07-15 cs.CL

DeltaMerge-LowRes: Composing Language and Task Deltas for Low-Resource Adaptation

Son Ha Xuan , Xuan-Bach Le , Phat T. Tran-Truong This is my paper
classification cs.CL
keywords model mergingcross-axis TIESlow-resource NLPlanguage adaptationtask adaptationweight-space compositioncalibrationAfrican languages
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Adapting a multilingual model to a new language and a new task usually means an expensive joint fine-tuning run for every language-task pair. This paper asks whether the two adaptations can be learned once and recombined in weight space: a language delta trained on unlabeled monolingual text plus a task delta trained on English labels, merged with a simple recipe. It introduces cross-axis TIES, which applies the trim, sign-elect, and merge steps of the TIES-Merging recipe to the language-task pair, and finds it beats task-only adaptation by +4 to +7 chrF (a character n-gram F-score for summarisation quality) on three of four African languages, and improves question-answering F1 and exact match. A sparsity-aware rule also reduces classification calibration error by 36 percent with no loss in accuracy. The paper's point is that the composition rule materially determines whether the merged model keeps the language signal, the task signal, or both.

Core claim

On the paper's own terms, the central discovery is that a separately trained language update and a separately trained task update can be composed in weight space, and the choice of composition rule decides whether the merged model keeps the language signal, the task signal, or both. Cross-axis TIES takes the established trim-sign-elect-merge recipe and applies it across the language and task axes: trim each delta to its top 20% magnitudes, elect one sign per coordinate, and keep only coordinates whose signs agree. Because the two deltas are trained on disjoint corpora, sign disagreement marks a coordinate where the language-fluency and task objectives conflict, so sign election acts as a str

What carries the argument

The central object is the cross-axis TIES composition rule, which adapts the TIES-Merging steps — trim each delta to its top-20% magnitudes, elect a sign per coordinate, merge only same-sign entries — from merging two task deltas to merging a language delta and a task delta. The deltas are low-rank (rank-16) updates to the base model's projections, trained once from unlabeled monolingual text and English labels, then materialized as dense tensors and combined by a dataless rule before evaluation. Sign election across axes acts as a structural filter for coordinates where language and task objectives conflict; the sparsity-aware variant instead masks to the top-20% of combined magnitudes and

Load-bearing premise

The headline summarisation gain compares three seeds for cross-axis TIES against only one seed for task-only and additive baselines, and the paper itself flags that the single seed could be unrepresentative.

What would settle it

Run task-only and additive summarisation baselines with the same three training seeds as cross-axis TIES; if the mean chrF gap across seeds falls to about the 1-point bootstrap CI width, the claimed +4 to +7 gain is not established. A matched-compute jointly fine-tuned baseline that clearly outperforms the composed model would also weaken the recovery claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • New language-task pairs can be adapted by composing preexisting deltas, eliminating the per-pair fine-tuning run for each pair.
  • For generation-heavy tasks, cross-axis TIES can recover a large part of the gap to task-only or additive baselines, including a 5.4x relative improvement on Amharic summarisation.
  • The composition rule materially changes behaviour on secondary metrics: NER recall rises about 3 points at a 2-point precision cost, and classification calibration improves by 36% ECE with no macro-F1 loss.
  • Because deltas are small and combined offline, the approach lowers the infrastructure and reproducibility cost of low-resource adaptation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the task-dependent ordering — generation > span extraction > classification/NER — holds beyond these four languages, composition rules could be selected by task type instead of per-pair, and the ordering itself becomes a testable taxonomy of language-task interaction.
  • The sign-election mechanism predicts that languages with noisier or smaller language deltas should show the largest gap between additive merging and cross-axis TIES; the Yoruba cell is one data point, and adding more languages with varied corpus sizes would test this directly.
  • The calibration gain from zeroing low-magnitude sign-conflicting coordinates may be a general phenomenon for merging any two deltas trained on different objectives, not just language and task; this could be probed in other multi-axis merging settings.
  • A matched-compute comparison against joint fine-tuning (e.g., continued pretraining plus supervised tuning on the same budget) would settle how much of end-to-end adaptation the modular route actually recovers; the paper leaves this explicitly out of scope.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies weight-space composition of a language delta (trained on unlabeled monolingual text) and a task delta (trained on English labels) for low-resource African-language adaptation. It compares four composition rules—additive, activation-guided, sparsity-aware, and the proposed cross-axis TIES—against a task-only baseline, holding (ΔL, ΔT) and the evaluation protocol fixed across 158 cells. The main claims are: cross-axis TIES improves summarisation chrF on 3/4 languages by +4–7 (chrF 18.59 vs. 13.80 for task-only); it improves QA F1 by +2.32 and EM by +2.91; and sparsity-aware merging reduces classification ECE by 36% at parity macro-F1. The paper releases JSON traces and a claim ledger and is explicit about bootstrap and seed limitations.

Significance. If the summarisation and calibration findings are robust, the paper makes a useful methodological contribution: it shows that the composition rule matters when merging language and task deltas, and cross-axis TIES is a sensible adaptation of TIES to the language–task plane. The experimental design is unusually controlled—identical delta tensors across rules, 10,000-sample paired bootstrap over test splits, fixed hyperparameters, and full traceability via the claim ledger. The main limitations are acknowledged internally, but they undercut the strength of the headline: the summarisation result rests on a 3-vs-1 seed comparison, the QA gains are below the per-cell CI width, and no matched-compute external baseline is reported. Thus the internal 'which rule?' question is sound, but the abstract and title risk overstating the practical 'low-resource adaptation' claim.

major comments (3)
  1. [§4.2, Tables 3 & 11] The headline summarisation result is a three-seed vs. one-seed comparison. Table 11 shows task-only and additive have seed 0 only, while cross-axis TIES has seeds 0–2; Table 3 reports Δ_TIES vs. task-only of +9.44, +3.89, +6.91, and −1.07. The bootstrap CIs (≈0.5 chrF) quantify only test-set resampling, not training-seed variance. Given observed seed SDs of ~1–2 points on the encoder tasks, the Hausa margin could shrink and Yoruba could flip. The paper should run at least three seeds for task-only and additive on summarisation, or downgrade the central claim to 'directional' and remove the 18.59 vs. 13.80 contrast from the abstract.
  2. [§4.2, Tables 1 & 10] The QA finding is not statistically supported as stated. The aggregate F1 gain of +2.32 is smaller than the per-cell bootstrap CI half-width of 4.0–4.5 F1 reported in Table 1. The win counts of 5/9 (F1) and 7/9 (EM) are descriptive, not inferential. The paper needs a proper paired analysis over language–seed cells (e.g., bootstrap over cell means or a mixed-effects model), or it must present the QA effect as a weak/directional trend in the abstract and §1 rather than as a headline improvement.
  3. [§3, §7, App. A] The internal five-recipe comparison is clean, but no matched-compute external baseline is run (target-language LoRA at b=256, Continue-PT+LoRA, or joint FT). The paper explicitly scopes this out, yet the title and introduction present the method as a low-resource adaptation recipe, and the practical value of 'zero per-pair training cost' cannot be assessed without at least one such baseline. I recommend adding one matched baseline on a subset of cells or substantially narrowing the framing to 'composition-rule comparison only.' As it stands, the significance claims exceed the evidence.
minor comments (6)
  1. [§2.2, Eq. (2)] The notation for additive and sparsity-aware rules is ambiguous: Eq. (2) uses (α, β), while Eq. (3) defines per-layer (α_ℓ, β_ℓ). State explicitly whether additive and sparse use global scalars, per-layer scalars, or a fixed default.
  2. [§4.2, Fig. 2(a)] The claim that the winning margin is '5–10× the per-cell CI width' is inaccurate for Amharic (≈19×) and Swahili (≈14×); either update the range or say 'at least 5×.'
  3. [§5.2, Table 7] The phrase 'SUM +4.79 (3/4 paired cells; single-seed comparators)' should be spelled out: because task-only has only seed 0, the comparison is not paired at the seed level; the 3/4 refers to languages, not paired cells.
  4. [Table 2 caption] The table row for activation-guided on SUM contains a dash, and the caption notes it was not run; this is fine, but add the same dash to the table body for consistency with the caption and with Appendix C.
  5. [Appendix D] The comparisons with published baselines are explicitly qualitative and uncontrolled. Add a one-sentence warning in the main text so readers do not treat these as head-to-head results.
  6. [Limitations] The sentence in the Limitations that the summarisation headline 'survives Bonferroni correction' may be misinterpreted because it refers only to test-set bootstrap variation, not training-seed variance. Soften it to match the seed caveat stated in the same paragraph.

Circularity Check

0 steps flagged

No significant circularity: the composition rules are evaluated with fixed or unlabeled-probe-derived hyperparameters on held-out test splits, and the paper explicitly flags its single-seed and post-hoc interpretation limits.

full rationale

I found no circular step. The central comparison holds (ΔL, ΔT) fixed and varies only the composition rule; none of the rules' hyperparameters are fit to target-language test labels. α and β come from the closed-form activation-shift formula on 50 unlabeled probe sentences (Eq. 3), k=20% is fixed, and every rule is evaluated on held-out test splits with released bootstrap traces. Thus the headline 'cross-axis TIES wins summarisation on 3/4 languages' is not a fitted quantity renamed as a prediction. The only circularity-adjacent items are explicitly disclaimed by the paper: (a) the summarisation comparison is 3-seed TIES vs. single-seed task-only/additive, which the paper flags as 'not yet checked against training-seed variance of the comparators' (§4.2, Limitations, Table 3) — this is a statistical robustness caveat, not a reduction of the result to its inputs; (b) the mechanistic ordering 'SUM ≫ QA > CLS/NER' is admitted to be 'consistent with our data but not independently testable from these experiments alone' (§2.2) — an interpretive hypothesis, not a load-bearing derived prediction. There are no self-citations by the authors; the TIES/task-arithmetic machinery is cited to external work (Yadav et al. 2023; Ilharco et al. 2023), and cross-axis TIES is a new application rather than a uniqueness claim. The lack of matched external baselines (LoRA, Continue-PT+LoRA, plain TIES) is acknowledged in §3, §6, and Limitations and affects external validity, not circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

No new physical or mathematical entities are introduced. The method relies on standard model-merging assumptions and a handful of hand-set hyperparameters; the most notable untested assumption is that the single-seed baseline is representative.

free parameters (3)
  • Sparsity fraction k = 20%
    Fixed by hand for all cells; no per-cell sweep. The top-k mask in Eq. 4 and the trim step in cross-axis TIES use this value; results could depend on it.
  • Activation-guided per-layer α_l, β_l = not reported in text; per-layer values in traces
    Computed from 50 unlabeled probe sentences (Eq. 3) and reused in sparsity-aware and cross-axis TIES. They are closed-form, not supervised, but they are still chosen scalars that affect the merge.
  • LoRA rank r = 16
    Delta parameterization; all results are conditional on this rank.
axioms (5)
  • domain assumption θ0 + ΔL + ΔT is a valid approximation of joint language–task fine-tuning.
    Underpins Eq. 1 and all composition rules; borrowed from model-merging literature, not proven for cross-axis deltas in this paper.
  • domain assumption A delta trained by MLM/span corruption on target-language text transfers useful fluency into the merged model.
    Central to the language axis; the Yoruba summarisation cell (additive loss, task-only leads) shows this can fail.
  • ad hoc to paper Per-layer activation shifts from 50 unlabeled probe sentences reliably weight language vs task importance.
    Eq. 3 defines α_l, β_l from probes; no robustness/convergence check, but no target labels are used.
  • domain assumption The AG News → MasakhaNEWS four-class mapping is a valid supervision bridge; unmapped classes are scored as zero-prediction.
    App B explains mapping; affects classification results and the calibration claim.
  • domain assumption Test labels are correct and test splits are not seen during delta training.
    Standard empirical assumption; necessary for any score to be meaningful.

pith-pipeline@v1.3.0-alltime-deepseek · 20411 in / 12800 out tokens · 117226 ms · 2026-08-02T03:09:43.953207+00:00 · methodology

0 comments
read the original abstract

Adapting a multilingual encoder to a new language \emph{and} a new task with only a few hundred gold examples is a common low-resource NLP setting, yet the two axes are usually fused via an expensive language--task fine-tuning run. We ask whether they can instead be trained separately and recombined in weight space. \DeltaMergeLowRes{} learns a language delta $\Delta_L$ from unlabeled monolingual text and a task delta $\Delta_T$ from labeled English data, then composes them at inference under one of four rules: additive, activation-guided, sparsity-aware, and a novel \emph{cross-axis TIES}. The new rule adapts the TIES-Merging steps of trimming, sign election, and merging to the language and task axes rather than to two task axes. Holding $(\Delta_L,\Delta_T)$ fixed across rules on four task families and four African languages ($158$ evaluated cells, $10{,}000$-sample paired bootstrap per cell), we find: (i) cross-axis TIES wins summarisation on $3/4$ languages by $+4$ to $+7$ chrF (chrF $18.59$ vs.\ $13.80$ task-only); (ii) it improves QA F1 by $+2.32$ and EM by $+2.91$; and (iii) sparsity-aware merging cuts classification ECE by $36\%$ at parity macro-F1. The composition rule materially changes what the merged model preserves, suppresses, and calibrates. We release all JSON traces and a claim ledger.

Figures

Figures reproduced from arXiv: 2607.13967 by Phat T. Tran-Truong, Son Ha Xuan, Xuan-Bach Le.

Figure 1
Figure 1. Figure 1: DeltaMerge-LowRes framework. A base multilingual model [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) Per-language chrF on summarisation. Cross-axis TIES wins three of four languages by +4 to +7 chrF; per-cell bootstrap CIs are about 1 chrF (Ta￾ble 11). (b) QA F1 and EM averaged over three encoder languages; cross-axis TIES tops both. than a fully matched 3×3 one. We flag this in the tables; the key follow-up is ≥ 3 task-only/additive seeds. Why Yoruba behaves differently. Yoruba is the one summarisati… view at source ↗
Figure 3
Figure 3. Figure 3: Calibration–accuracy plane on classifica [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 3 canonical work pages

  1. [1]

    David Ifeoluwa Adelani, Marek Masiak, Israel Abebe Azime, Jesujoba Alabi, Atnafu Lambebo Tonja, Christine Mwase, Odunayo Ogundepo, Bonaventure F. P. Dossou, Akintunde Oladipo, Doreen Nixdorf, Chris Chinenye Emezue, Sana Al-azzawi, Blessing Sibanda, Davis David, Lolwethu Ndolela, Jonathan Mukiibi, Tunde Ajayi, Tatiana Moteu, Brian Odhiambo, and 46 others. ...

    doi:10.18653/v1/2023.ijcnlp-main.10 2023

  2. [2]

    Alabi, Shamsuddeen H

    David Ifeoluwa Adelani, Graham Neubig, Sebastian Ruder, Shruti Rijhwani, Michael Beukman, Chester Palen-Michel, Constantine Lignos, Jesujoba O. Alabi, Shamsuddeen H. Muhammad, Peter Nabende, Cheikh M. Bamba Dione, Andiswa Bukula, Rooweither Mabuya, Bonaventure F. P. Dossou, Blessing Sibanda, Happy Buzaaba, Jonathan Mukiibi, Godson Kalipe, Derguene Mbaye, ...

    doi:10.18653/v1/2022.emnlp-main.298 2022

  3. [3]

    Alan Ansell, Edoardo Maria Ponti, Anna Korhonen, and Ivan Vuli \'c . 2022. https://doi.org/10.18653/v1/2022.acl-long.125 Composable sparse fine-tuning for cross-lingual transfer . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1778--1796, Dublin, Ireland. Association for Computatio...

    doi:10.18653/v1/2022.acl-long.125 2022

  4. [4]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning at scale . In Proceedings of the 58th Annual Meeting of the Association for Comp...

    doi:10.18653/v1/2020.acl-main.747 2020

  5. [5]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. https://doi.org/10.52202/075280-0441 QLoRA : Efficient finetuning of quantized LLMs . In Advances in Neural Information Processing Systems, volume 36, pages 10088--10115. Curran Associates, Inc

    doi:10.52202/075280-0441 2023

  6. [6]

    Rush, and Yoon Kim

    Demi Guo, Alexander M. Rush, and Yoon Kim. 2021. https://doi.org/10.18653/v1/2021.acl-long.378 Parameter-efficient transfer learning with diff pruning . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4884--4...

    doi:10.18653/v1/2021.acl-long.378 2021

  7. [7]

    Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M

    Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M. Sohel Rahman, and Rifat Shahriyar. 2021. https://doi.org/10.18653/v1/2021.findings-acl.413 XL -sum: Large-scale multilingual abstractive summarization for 44 languages . In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4...

    doi:10.18653/v1/2021.findings-acl.413 2021

  8. [8]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. https://proceedings.mlr.press/v97/houlsby19a.html Parameter-efficient transfer learning for NLP . In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine ...

    2019

  9. [9]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://doi.org/10.48550/arXiv.2106.09685 LoRA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    Pith/arXiv arXiv doi:10.48550/arxiv.2106.09685 2022

  10. [10]

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. https://doi.org/10.48550/arXiv.2212.04089 Editing models with task arithmetic . In International Conference on Learning Representations

    Pith/arXiv arXiv doi:10.48550/arxiv.2212.04089 2023

  11. [11]

    Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. 2023. https://doi.org/10.48550/arXiv.2212.09849 Dataless knowledge fusion by merging weights of language models . Preprint, arXiv:2212.09849

    Pith/arXiv arXiv doi:10.48550/arxiv.2212.09849 2023

  12. [12]

    Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. https://doi.org/10.18653/v1/E17-2068 Bag of tricks for efficient text classification . In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427--431. Association for Computational Linguistics

    doi:10.18653/v1/e17-2068 2017

  13. [13]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.243 The power of scale for parameter-efficient prompt tuning . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045--3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics

    doi:10.18653/v1/2021.emnlp-main.243 2021

  14. [14]

    Xiang Lisa Li and Percy Liang. 2021. https://doi.org/10.18653/v1/2021.acl-long.353 Prefix-tuning: Optimizing continuous prompts for generation . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582--4597, Onl...

    doi:10.18653/v1/2021.acl-long.353 2021

  15. [15]

    Michael Matena and Colin Raffel. 2022. https://doi.org/10.48550/arXiv.2111.09832 Merging models with fisher-weighted averaging . Preprint, arXiv:2111.09832

    Pith/arXiv arXiv doi:10.48550/arxiv.2111.09832 2022

  16. [16]

    Gwadabe, Clara E

    Odunayo Ogundepo, Tajuddeen R. Gwadabe, Clara E. Rivera, Jonathan H. Clark, Sebastian Ruder, David Ifeoluwa Adelani, Bonaventure F. P. Dossou, Abdou Aziz Diop, Claytone Sikasote, Gilles Hacheme, Happy Buzaaba, Ignatius Ezeani, Rooweither Mabuya, Salomey Osei, Chris Emezue, Albert Njoroge Kahira, Shamsuddeen Hassan Muhammad, Akintunde Oladipo, Abraham Tolu...

    work page doi:10.18653/v1/2023.findings-emnlp.997 2023

  17. [17]

    Pedro Javier Ortiz Su \'a rez, Laurent Romary, and Beno \^i t Sagot. 2020. https://doi.org/10.18653/v1/2020.acl-main.156 A monolingual approach to contextualized word embeddings for mid-resource languages . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1703--1714, Online. Association for Computational Li...

    work page doi:10.18653/v1/2020.acl-main.156 2020

  18. [18]

    Jonas Pfeiffer, Aishwarya Kamath, Andreas R \"u ckl \'e , Kyunghyun Cho, and Iryna Gurevych. 2021. https://doi.org/10.18653/v1/2021.eacl-main.39 Adapterfusion: Non-destructive task composition for transfer learning . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487--503,...

    doi:10.18653/v1/2021.eacl-main.39 2021

  19. [19]

    Jonas Pfeiffer, Ivan Vuli \'c , Iryna Gurevych, and Sebastian Ruder. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.617 MAD-X : A n A dapter- B ased F ramework for M ulti- T ask C ross- L ingual T ransfer . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7654--7673, Online. Association for Computational ...

    doi:10.18653/v1/2020.emnlp-main.617 2020

  20. [20]

    Maja Popovi \'c . 2015. https://doi.org/10.18653/v1/W15-3049 chr F : character n-gram F -score for automatic MT evaluation . In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392--395. Association for Computational Linguistics

    doi:10.18653/v1/w15-3049 2015

  21. [21]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. https://doi.org/10.18653/v1/D16-1264 SQ u AD : 100,000+ questions for machine comprehension of text . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383--2392. Association for Computational Linguistics

    doi:10.18653/v1/d16-1264 2016

  22. [22]

    Yi-Lin Sung, Varun Nair, and Colin Raffel. 2021. https://doi.org/10.48550/arXiv.2111.09839 Training neural networks with fixed sparse masks . Preprint, arXiv:2111.09839

    Pith/arXiv arXiv doi:10.48550/arxiv.2111.09839 2021

  23. [23]

    Tjong Kim Sang and Fien De Meulder

    Erik F. Tjong Kim Sang and Fien De Meulder. 2003. https://doi.org/10.18653/v1/W03-0419 Introduction to the C o NLL -2003 shared task: Language-independent named entity recognition . In Proceedings of the Seventh Conference on Natural Language Learning at HLT - NAACL 2003 , pages 142--147

    work page doi:10.18653/v1/w03-0419 2003

  24. [24]

    Ahmet \"U st \"u n, Arianna Bisazza, Gosse Bouma, Gertjan van Noord, and Sebastian Ruder. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.541 Hyper-x: A unified hypernetwork for multi-task multilingual transfer . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7934--7949, Abu Dhabi, United Arab Emirates. ...

    doi:10.18653/v1/2022.emnlp-main.541 2022

  25. [25]

    Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt

    Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. 2022 a . https://proceedings.mlr.press/v162/wortsman22a.html Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing infe...

    2022

  26. [26]

    Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. 2022 b . https://doi.org/10.48550/arXiv.2109.01903 Robust fine-tuning of zero-shot models . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    Pith/arXiv arXiv doi:10.48550/arxiv.2109.01903 2022

  27. [27]

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. https://doi.org/10.18653/v1/2021.naacl-main.41 m T 5: A massively multilingual pre-trained text-to-text transformer . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics...

    doi:10.18653/v1/2021.naacl-main.41 2021

  28. [28]

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. https://doi.org/10.48550/arXiv.2306.01708 TIES -merging: Resolving interference when merging models . Preprint, arXiv:2306.01708

    Pith/arXiv arXiv doi:10.48550/arxiv.2306.01708 2023

  29. [29]

    Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. https://proceedings.neurips.cc/paper/2015/file/250cf8b51c773f3f8dc8b4be867a9a02-Paper.pdf Character-level convolutional networks for text classification . In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc

    2015