Pith. sign in

REVIEW 3 major objections 5 minor 36 references

DeFTX: Denoised Sparse Fine-Tuning for Zero-Shot Cross-Lingual Transfer

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SVD-denoised sparse fine-tuning improves zero-shot cross-lingual transfer.

desk verdict A modest, clearly-described extension of LT-SFT with a useful ablation table, but the paper's own ablation undercuts its denoising story and the gains are small and statistically untested. read the letter →

arxiv 2505.15090 v1 pith:42SIDN3S submitted 2025-05-21 cs.CL cs.AIcs.LG

Sona Elza Simon , Preethi Jyothi This is my paper
classification cs.CLcs.AIcs.LG
keywords zero-shotcross-lingualtransfersparsefine-tuningsingularvaluedecompositionlow-rankdenoisinglotterytickethypothesisNusaXAmericasNLItaskarithmetic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces DeFT-X, a composable sparse fine-tuning method for zero-shot cross-lingual transfer. It claims that applying singular value decomposition to denoise the weight-update matrix before magnitude-based pruning yields sparse fine-tuned vectors that transfer better to low-resource languages when composed. On NusaX sentiment analysis, DeFT-X reaches an average F1 of 81.4 versus 80.2 for the LT-SFT baseline, and on AmericasNLI it reaches 51.3 accuracy versus 51.0. The practical payoff would be a small but consistent way to improve modular, parameter-efficient transfer to languages with no labeled data.

What carries the argument

The load-bearing object is the SVD-denoised weight update: for each matrix $W$ in $\Delta W$, form $L = U_r \Sigma_r V_r^T$ from the top-$r$ singular components, keep a mask $m$ over the top $n$ entries of $|W-L|$, set $S = m \odot (W-L)$, and replace $W$ by $L+S$. This is followed by magnitude pruning of the entire denoised $\Delta W$ and a sparse fine-tuning pass that resets to $\theta^{(0)}$ and trains only the masked parameters. The mechanism is intended to separate high-singular-value structure from low-singular-value noise while retaining a sliver of the noise that the ablation shows is still useful, and it reduces overlap between language and task vectors.

What would settle it

Run DeFT-X on NusaX with the retention fraction for higher-order components set to 0%, 1%, 5%, 10%, and 20% while holding everything else fixed. If the average F1 does not peak near 5% and exceed LT-SFT's 80.2, then the paper's signal-noise narrative is not what carries the result.

Watch

Extended reading notes

Core claim

DeFT-X's central claim is that the difference between a fully fine-tuned model and the pretrained model, $\Delta W = \theta^{(1)} - \theta^{(0)}$, contains noisy higher-order components that degrade the sparse vectors used for transfer. Each weight matrix in $\Delta W$ is decomposed as $W = U\Sigma V^T$, keeping a low-rank part $L$ built from the largest singular values, then magnitude-pruning the residual $W-L$ down to a small fraction (5%) rather than discarding it outright. The reconstructed, denoised $\Delta W$ is then magnitude-pruned and sparsely fine-tuned to produce task-specific and language-specific vectors that are composed by addition, $\theta_{TL} = \theta^{(0)} + \phi_T + \phi_L$. In the paper's best configurations this improves zero-shot transfer over LT-SFT on both benchmarks, including a gain of 1.2 F1 points on NusaX and 0.3 accuracy points on AmericasNLI.

Load-bearing premise

The load-bearing premise is that low singular value components of the weight update are mostly noise, yet the paper's own ablation shows that removing them outright hurts performance, so the benefit actually depends on the unexamined decision to keep 5% of them.

Editorial extensions

If this is right

  • On NusaX, the best DeFT-X configuration raises average F1 from 80.2 (LT-SFT) to 81.4 with XLM-R base, and improves several individual languages such as Minangkabau.
  • On AmericasNLI, DeFT-X with $r_l = 200$ and $r_t = 90\%$ variance reaches 51.3 average accuracy versus 51.0 for LT-SFT.
  • DeFT-X preserves performance when task vectors are trained without source-language initialization, whereas LT-SFT degrades, suggesting the denoised vectors are more robust to composition.
  • Parameter overlap between language and task vectors drops with DeFT-X, implying less destructive interference when vectors are added.
  • DeFT-X also improves over LT-SFT on the mE5 base embedding model (from 72.1 to 74.0 average F1), indicating the benefit is not tied to XLM-R alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The improvement may owe more to the fixed 5% retention fraction and to the extra magnitude-pruning step than to a genuine separation of signal from noise: the paper's own ablation shows that removing the higher-order components entirely costs 0.8 F1 points.
  • If the 5% fraction is a hidden tuning knob, then a testable consequence is that the gain over LT-SFT should vary with that fraction; a sweep over retention rates would clarify whether denoising or the chosen fraction drives the result.
  • Because rank selection is manual and model-specific, a data-driven criterion (for example, choosing rank by validation performance on the source language) would turn DeFT-X from a recipe into a principle.
  • The same denoising step should apply to task arithmetic beyond cross-lingual transfer, e.g., merging models for multi-task learning, where noisy updates are also composed additively; the authors hint at this but do not test it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DeFT-X, a modification of LT-SFT for zero-shot cross-lingual transfer. Before magnitude pruning and sparse fine-tuning, DeFT-X applies an SVD-based denoising step to the weight update matrix ΔW: each matrix is replaced by L + S, where L keeps the top-r singular components and S retains the top 5% of the residual by magnitude. Language-specific and task-specific sparse vectors are then learned as in LT-SFT and composed by addition. The method is evaluated on NusaX sentiment analysis (five Indonesian languages) and AmericasNLI (ten indigenous languages) with XLM-R base/large and mE5 base, reporting average F1 and accuracy. The best reported configurations outperform LT-SFT by 1.2 F1 on NusaX and 0.3 accuracy on AmericasNLI with XLM-R base.

Significance. If the results are robust, DeFT-X is a simple, modular improvement over LT-SFT that can be dropped into existing sparse fine-tuning pipelines, and the paper's focus on truly low-resource languages is valuable. The paper is clearly written, reports per-language numbers, and includes ablations isolating the contributions of higher-order components, magnitude pruning, and sparse fine-tuning. The contribution is empirical rather than theoretical, and the claimed gains are small, so the evidence must be stronger than a single run per configuration. The explicit admission in the Limitations section that the SVD rank requires manual tuning is honest but also highlights that the headline numbers are partially configuration-selected.

major comments (3)
  1. [§3.1, §4.2, Table 5] The motivation for SVD denoising is that higher-order (lower singular-value) components are 'more likely to capture uninformative or noisy artefacts' and should be pruned. Table 5 contradicts this: removing the retained higher-order components reduces NusaX average F1 from 81.4 to 80.6, and the text in §5 states that 'higher-order components contain useful information, making it essential to retain them after denoising.' Since the total gain over LT-SFT is 81.4 vs 80.2 (1.2 F1), the improvement could come entirely from the choice to retain 5% of the residual rather than from a principled signal-noise separation. The retention fraction is never varied and no evidence is given that 5% generalizes across languages, tasks, or model sizes. Please add an experiment sweeping n (e.g., 0%, 1%, 5%, 10%, 25%) and a control that retains random or smallest-magnitude residual entries, and report whether the benefit is monotone and where it saturates.
  2. [§4.2, §5, Tables 2 and 3] The paper reports a single run per configuration and selects the best among several rank combinations (r_l, r_t) separately for each task and model size, yet the claimed improvements are small, especially on AmericasNLI (51.3 vs 51.0 average accuracy). With only 750 test examples per language and no error bars, confidence intervals, or significance tests, these differences are compatible with noise. In addition, the uniform rank values are described as 'selected by observing the overall rank that covers 90% variance,' with no held-out validation described, so the reported best numbers are optimistic. Please report multiple seeds or bootstrap confidence intervals, paired significance tests for per-language differences, and a rank-selection procedure on a validation split; without this, the abstract's 'at par or outperforms' claim is not supported.
  3. [§3.2, §4.2, Appendix A] There is a discrepancy about which source-language vector is used during task-vector training. Section 4.2 states that for XLM-R base, task adaptation 'always apply[ies] the source language SFT from LT-SFT,' while §3.2 and Algorithm 1 (Appendix A, line 24) describe DeFT-X as computing its own source-language vector via DEFT-X. If the reported DeFT-X task vectors were trained with an LT-SFT source vector, the final composed model is not purely DeFT-X, and the robustness claim in Table 4 is hard to interpret. Please clarify which source vector was used for each table, or rerun the base-model experiments with DeFT-X's own source vector.
minor comments (5)
  1. [Eq. (4)] The symbol n is used both for the column dimension of W ∈ R^{m×n} and for the number of retained entries in the top-n mask; please rename one of them to avoid ambiguity.
  2. [Figure 2 and Figure 3] The claim that DEFT-X 'results in lower overlap' is made without quantitative support; please report mean overlap, standard deviations, and the number of tasks/languages averaged over.
  3. [§3.1] The terms 'lower-order components' and 'higher-order components' are nonstandard and are used in the opposite direction of the usual spectral terminology; please define them explicitly at first use.
  4. [Appendix E] There is a typo in 'initilizalized'; it should be 'initialized.'
  5. [§4.2, general] The paper does not mention whether code will be released; given the number of hyperparameters and configuration choices, a public implementation would substantially aid reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DeFT-X is evaluated on held-out public benchmark test sets against external baselines, and its derivation does not reduce to its own inputs.

full rationale

The paper's core algorithm is an empirical modification of LT-SFT: it computes the weight update delta = theta^(1) - theta^(0), applies SVD to each matrix, retains a low-rank approximation L, keeps a magnitude-pruned residual S from the higher-order components, reconstructs W as L + S, then applies magnitude pruning and sparse fine-tuning. This is a defined procedure, not a derivation that assumes its own conclusion. Evaluation is performed on held-out test sets from NusaX and AmericasNLI, and the reported numbers are compared against LT-SFT and MAD-X baselines. The internal hyperparameters (rank r and the 5% retention fraction) are manually chosen and tuned, but the test-set labels are independent of the hyperparameter selection, so no fitted parameter is being renamed as a prediction. The paper cites prior work for SVD denoising and LT-SFT, but these are external citations, not self-citations, and no uniqueness theorem or author-imported constraint is used to force the method's choice. The ablation in Table 5 does contradict the paper's 'higher-order components are noise' motivation by showing that retaining 5% of them helps, but this is an internal consistency and hyperparameter-justification issue, not a circularity in the formal sense defined here. Because the central claim is tested against external benchmarks and the derivation chain does not reduce to its inputs, the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

DeFT-X introduces no new entities; it relies on standard SVD and pruning heuristics. The free parameters are largely inherited from LT-SFT, with the novel choices being the SVD rank and the retention fraction.

free parameters (4)
  • SVD rank r_l and r_t = 100/200/300 or 90% variance, varying per model and task
    Chosen empirically; the paper tests multiple values and highlights the best. The Limitations state the rank requires manual tuning.
  • Higher-order retention fraction n = 5% of higher-order components
    Fixed heuristic used in the mask m over (W-L); no sensitivity analysis is provided.
  • Sparse mask size k = 7.6M (language), 14.2M (task)
    Inherited from LT-SFT to match adapter parameter counts; central to the method but not tuned here.
  • L1 regularization lambda = 0.1
    Used for language adaptation MLM training; inherited from LT-SFT setup.
assumptions (4)
  • domain assumption Magnitude pruning identifies effective sparse subnetworks
    DeFT-X relies on the lottery ticket hypothesis to select masks after denoising; inherited from LT-SFT.
  • domain assumption SVD low-rank components separate signal from noise in weight updates
    The method assumes high singular values capture useful structure and low singular values are noise, citing LASER and LoRS-Merging; this is a heuristic and is partially contradicted by the paper's own ablation.
  • domain assumption Composing language and task vectors via addition avoids destructive interference
    Inherited from LT-SFT and task arithmetic; the paper provides some overlap analysis but no guarantee.
  • domain assumption NusaX and AmericasNLI are representative zero-shot cross-lingual benchmarks
    Standard benchmarks, but limited to two classification tasks and encoder-only models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeFTX: Denoised Sparse Fine-Tuning for Zero-Shot Cross-Lingual Transfer." pith.science (2026). https://pith.science/paper/42SIDN3S

@misc{pith2026250515090,
  author       = {Pith},
  title        = {Pith review of: DeFTX: Denoised Sparse Fine-Tuning for Zero-Shot Cross-Lingual Transfer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/42SIDN3S}},
  note         = {Machine review of arXiv:2505.15090}
}
read the original abstract

Effective cross-lingual transfer remains a critical challenge in scaling the benefits of large language models from high-resource to low-resource languages. Towards this goal, prior studies have explored many approaches to combine task knowledge from task-specific data in a (high-resource) source language and language knowledge from unlabeled text in a (low-resource) target language. One notable approach proposed composable sparse fine-tuning (SFT) for cross-lingual transfer that learns task-specific and language-specific sparse masks to select a subset of the pretrained model's parameters that are further fine-tuned. These sparse fine-tuned vectors (SFTs) are subsequently composed with the pretrained model to facilitate zero-shot cross-lingual transfer to a task in a target language, using only task-specific data from a source language. These sparse masks for SFTs were identified using a simple magnitude-based pruning. In our work, we introduce DeFT-X, a novel composable SFT approach that denoises the weight matrices of a pretrained model before magnitude pruning using singular value decomposition, thus yielding more robust SFTs. We evaluate DeFT-X on a diverse set of extremely low-resource languages for sentiment classification (NusaX) and natural language inference (AmericasNLI) and demonstrate that it performs at par or outperforms SFT and other prominent cross-lingual transfer baselines.

Figures

Figures reproduced from arXiv: 2505.15090 by the authors.

Figure 1
Figure 1. A graphical representation of DEFT-X. The pretrained model θ (gray, left) undergoes full fine-tuning to obtain θFFT. The difference ∆W (blue and red, left) captures the magnitude difference between θ and θFFT. Each weight matrix in ∆W is denoised by pruning higher-order components (i.e., lower singular value components) while retaining lower-order components (i.e., high singular value components). The denoised ∆W is… view at source ↗
Figure 2
Figure 2. Comparing the overlap between the sparse [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 9 canonical work pages

  1. [1]

    Alan Ansell, Marinela Parovi \'c , Ivan Vuli \'c , Anna Korhonen, and Edoardo Ponti. 2023 a . https://doi.org/10.18653/v1/2023.emnlp-main.242 Unifying cross-lingual transfer across scenarios of resource scarcity . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3980--3995, Singapore. Association for Computa...

    work page doi:10.18653/v1/2023.emnlp-main.242 2023

  2. [2]

    Alan Ansell, Edoardo Ponti, Anna Korhonen, and Ivan Vuli \'c . 2022. https://doi.org/10.18653/v1/2022.acl-long.125 Composable sparse fine-tuning for cross-lingual transfer . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1778--1796, Dublin, Ireland. Association for Computational Li...

    doi:10.18653/v1/2022.acl-long.125 2022

  3. [3]

    Alan Ansell, Edoardo Maria Ponti, Anna Korhonen, and Ivan Vuli \'c . 2023 b . https://doi.org/10.18653/v1/2023.findings-acl.517 Distilling efficient language-specific models for cross-lingual transfer . In Findings of the Association for Computational Linguistics: ACL 2023, pages 8147--8165, Toronto, Canada. Association for Computational Linguistics

    work page doi:10.18653/v1/2023.findings-acl.517 2023

  4. [4]

    Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glava s , Ivan Vuli \'c , and Anna Korhonen. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.410 MAD - G : M ultilingual adapter generation for efficient cross-lingual transfer . In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4762--4781, Punta C...

    doi:10.18653/v1/2021.findings-emnlp.410 2021

  5. [5]

    Alan Ansell, Ivan Vulić, Hannah Sterz, Anna Korhonen, and Edoardo M. Ponti. 2024. https://arxiv.org/abs/2401.16405 Scaling sparse fine-tuning to large language models . Preprint, arXiv:2401.16405

    arXiv 2024

  6. [6]

    Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, Nayan Singhal, Hongjiang Lv, and Bing Liu. 2024. https://arxiv.org/abs/2410.01335 Layer swapping for zero-shot cross-lingual transfer in large language models . Preprint, arXiv:2410.01335

    arXiv 2024

  7. [7]

    Tyler Chang, Zhuowen Tu, and Benjamin Bergen. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.9 The geometry of multilingual language model representations . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 119--136, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics

    doi:10.18653/v1/2022.emnlp-main.9 2022

  8. [8]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. https://doi.org/10.18653/v1/D18-1269 XNLI : Evaluating cross-lingual sentence representations . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475--2485, Brussels, Belgium. Association...

    doi:10.18653/v1/d18-1269 2018

Show all 36 references
  1. [9]

    Abteen Ebrahimi, Manuel Mager, Arturo Oncevay, Vishrav Chaudhary, Luis Chiruzzo, Angela Fan, John Ortega, Ricardo Ramos, Annette Rios, Ivan Vladimir Meza Ruiz, Gustavo Gim \'e nez-Lugo, Elisabeth Mager, Graham Neubig, Alexis Palmer, Rolando Coto-Solano, Thang Vu, and Katharina...

    2022 doi

  2. [10]

    Jonathan Frankle and Michael Carbin. 2019. https://openreview.net/forum?id=rJl-b3RcF7 The lottery ticket hypothesis: Finding sparse, trainable neural networks . In International Conference on Learning Representations

    2019

  3. [11]

    Demi Guo, Alexander Rush, and Yoon Kim. 2021. https://doi.org/10.18653/v1/2021.acl-long.378 Parameter-efficient transfer learning with diff pruning . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Con...

    2021 doi

  4. [12]

    Song Han, Huizi Mao, and William J. Dally. 2016. http://arxiv.org/abs/1510.00149 Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding . In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto R...

    2016 arXiv

  5. [13]

    Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning both weights and connections for efficient neural networks. In Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1, NIPS'15, page 1135–1143, Cambridge, MA, USA...

    2015

  6. [14]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. https://arxiv.org/pdf/2106.09685v1/1000 Lora: Low-rank adaptation of large language models. ICLR, 1(2):3

    2022 arXiv

  7. [15]

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. https://arxiv.org/abs/2212.04089 Editing models with task arithmetic . Preprint, arXiv:2212.04089

    2023 arXiv

  8. [16]

    Eran Malach, Gilad Yehudai, Shai Shalev-Schwartz, and Ohad Shamir. 2020. Proving the lottery ticket hypothesis: Pruning is all you need. In International Conference on Machine Learning, pages 6682--6691. PMLR

    2020

  9. [17]

    Ashwinee Panda, Berivan Isik, Xiangyu Qi, Sanmi Koyejo, Tsachy Weissman, and Prateek Mittal. 2024. https://arxiv.org/abs/2406.16797 Lottery ticket adaptation: Mitigating destructive interference in llms . Preprint, arXiv:2406.16797

    2024 arXiv

  10. [18]

    Marinela Parovi \'c , Ivan Vuli \'c , and Anna Korhonen. 2024. https://aclanthology.org/2024.eacl-short.12/ Investigating the potential of task arithmetic for cross-lingual transfer . In Proceedings of the 18th Conference of the European Chapter of the Association for Computat...

    2024

  11. [19]

    Jonas Pfeiffer, Aishwarya Kamath, Andreas R \"u ckl \'e , Kyunghyun Cho, and Iryna Gurevych. 2021 a . https://doi.org/10.18653/v1/2021.eacl-main.39 A dapter F usion: Non-destructive task composition for transfer learning . In Proceedings of the 16th Conference of the European ...

    2021 doi

  12. [20]

    Jonas Pfeiffer, Sebastian Ruder, Ivan Vulić, and Edoardo Maria Ponti. 2024. https://arxiv.org/abs/2302.11529 Modular deep learning . Preprint, arXiv:2302.11529

    2024 arXiv

  13. [21]

    Jonas Pfeiffer, Ivan Vuli \'c , Iryna Gurevych, and Sebastian Ruder. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.617 MAD-X : A n A dapter- B ased F ramework for M ulti- T ask C ross- L ingual T ransfer . In Proceedings of the 2020 Conference on Empirical Methods in Natur...

    2020 doi

  14. [22]

    Jonas Pfeiffer, Ivan Vuli \'c , Iryna Gurevych, and Sebastian Ruder. 2021 b . https://doi.org/10.18653/v1/2021.emnlp-main.800 UNK s everywhere: A dapting multilingual language models to new scripts . In Proceedings of the 2021 Conference on Empirical Methods in Natural Languag...

    2021 doi

  15. [23]

    Ayu Purwarianti and Ida Ayu Putu Ari Crisdayanti. 2019. https://doi.org/10.1109/icaicta.2019.8904199 Improving bi-lstm performance for indonesian sentiment analysis using paragraph vector . In 2019 International Conference of Advanced Informatics: Concepts, Theory and Applicat...

    2019

  16. [24]

    Ash, and Dipendra Misra

    Pratyusha Sharma, Jordan T. Ash, and Dipendra Misra. 2023. https://arxiv.org/abs/2312.13558 The truth is in there: Improving reasoning in language models with layer-selective rank reduction . Preprint, arXiv:2312.13558

    2023 arXiv

  17. [25]

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2023. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695

    2023 arXiv

  18. [26]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. https://arxiv.org/abs/2402.05672 Multilingual e5 text embeddings: A technical report . Preprint, arXiv:2402.05672

    2024 arXiv

  19. [27]

    Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, and Ayu Purwarianti. 2020. https://doi.org/10.18653/v1/2020.aacl-main.85 I ndo NLU : Benchmark and resources for evalu...

    2020 doi

  20. [28]

    Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computation...

    2018 doi

  21. [29]

    Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya, Rahmad Mahendra, Fajri Koto, Ade Romadhony, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Pascale Fung, Timothy Baldwin, Jey Han Lau, Rico Sennrich, and Sebastian Ruder. 2023. https://doi.org/10.18653/v1/2023.eac...

    2023 doi

  22. [30]

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/1644c9af28ab7916874f6fd6228a9bcf-Paper-Conference.pdf Ties-merging: Resolving interference when merging models . In Advances in Neural I...

    2023

  23. [31]

    Tien-Ju Yang, Yu-Hsin Chen, and Vivienne Sze. 2017. https://doi.org/10.1109/CVPR.2017.643 Designing energy-efficient convolutional neural networks using energy-aware pruning . In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6071--6079

    2017 doi

  24. [32]

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: absorbing abilities from homologous models as a free lunch. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org

    2024

  25. [33]

    Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. 2022. https://arxiv.org/abs/2106.10199 Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models . Preprint, arXiv:2106.10199

    2022

  26. [34]

    Qiuming Zhao, Guangzhi Sun, Chao Zhang, Mingxing Xu, and Thomas Fang Zheng. 2025. https://arxiv.org/abs/2502.17380 Low-rank and sparse model merging for multi-lingual speech recognition and translation . Preprint, arXiv:2502.17380

    2025 arXiv

  27. [35]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  28. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.