Pith. sign in

REVIEW 3 major objections 8 minor 46 references

Rethinking Expert Training for Model Merging with Prompt Learning

T0 review · 3 major / 8 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training experts in two stages—prompts first, then the vision encoder—makes them merge into stronger multi-task models without changing the merger.

desk verdict Real training-side lever for mergeability with broad tables, but the headline gains partly ride on a better text interface that the Vision-FT baseline never gets. read the letter →

arxiv 2607.24465 v1 pith:DMEKZWWH submitted 2026-07-27 cs.CV

Christos Georgakilas , Aniello Panariello , Samir El Karrat Moreno , Simone Calderara , Dimosthenis Karatzas , Joost van de Weijer This is my paper · ORCID
classification cs.CV
keywords modelmergingpromptlearningCLIPtaskvectorsDual-TunedExpertsLoRAmulti-taskvision-languagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Model merging usually treats expert training as fixed and tries to fix interference only at merge time. This paper argues the opposite: how you train each expert strongly shapes how well those experts combine. It first shows that learned text prompts alone, with a frozen CLIP backbone, already give competitive multi-task accuracy because no backbone weights need to be merged. Building on that, Dual-Tuned Experts (DTEs) learn those prompts first, then fine-tune the vision encoder with prompts held fixed. The second stage still reaches full expert accuracy, but the parameter updates stay smaller and closer to the pretrained model, so standard mergers (weight averaging, task arithmetic, TIES, TSV-M, Iso-C) recover more per-task performance. The gains hold for full fine-tuning and LoRA, across CLIP sizes and 8/14/20-task suites, and even when DTE and ordinary experts are mixed in one merge.

What carries the argument

Dual-Tuned Experts (DTEs): a two-stage recipe that first absorbs task semantics into a small set of learned context tokens, then fine-tunes the vision encoder against those fixed prompts, shrinking task-vector magnitude and representation drift so linear and spectral mergers interfere less.

What would settle it

On the same 8/14/20-task CLIP suites, train matched DTE and Vision-FT experts to equal separate accuracy, merge with the same operator and lambda protocol, and check whether DTE merges no longer beat Vision-FT merges—or whether the cosine-similarity and interpolation-flatness gaps disappear.

Watch

Extended reading notes

Core claim

Dual-Tuned Experts—Stage-1 unified-context prompt learning with a frozen backbone, then Stage-2 vision-encoder fine-tuning with those prompts fixed—match separate-expert accuracy of ordinary one-stage vision fine-tuning while consistently raising merged multi-task accuracy under the same merging operators on CLIP vision benchmarks.

Load-bearing premise

At test time the system already knows which task is running, so it can pick the right learned prompt set (or hand-crafted templates) without merging or guessing prompts.

Editorial extensions

If this is right

  • Expert-training recipe becomes a first-class knob for merge quality, orthogonal to designing new merge operators.
  • Simple weight averaging of DTEs can beat advanced mergers of ordinary full fine-tunes when many tasks are combined.
  • The same two-stage idea transfers to LoRA experts and reduces the gap between naive full-space LoRA merges and core-space methods.
  • Partially replacing ordinary experts with DTEs in a mixed pool steadily raises merge accuracy, so partial adoption still helps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If task identity were unknown, one would still need a cheap task router or a way to merge prompt sets; the paper’s multi-task numbers assume that router is free.
  • The pattern suggests other PEFT first stages (adapters, visual prompts) might similarly shrink backbone drift before a second fine-tune, not only text prompts on CLIP.
  • Platforms that ship only weight deltas may need a lightweight side-channel for per-task prompts if DTE-style experts become common.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper argues that expert training — not just the merging operator — is a design axis for model merging. It first observes that per-task prompt learning on a frozen CLIP backbone, with prompts selected at inference, is already competitive with state-of-the-art weight-merging methods (Fig. 2), especially in the many-task regime. It then introduces Dual-Tuned Experts (DTEs): Stage 1 learns unified-context prompts with the backbone frozen; Stage 2 fine-tunes the vision encoder with prompts fixed. Across ViT-B/32 and ViT-L/14, 8/14/20-task CLIP benchmarks, full FT and LoRA experts, and five merging methods (WA, TA, TIES, TSV-M, Iso-C), merged DTEs outperform merged one-stage Vision-FT experts while separate-expert accuracy is matched (Tables 1-2). Mechanistic analyses report smaller representation drift (~64% relative cosine-similarity improvement to the pretrained backbone, Fig. 4), smaller Frobenius update norms (2.27 vs 2.58, paired tests p=0.025/0.001, §4.3), and flatter pairwise interpolation profiles (Fig. 5). A heterogeneous experiment shows monotonic improvement as Vision-FT experts are replaced by DTEs (Fig. 6), and a FLOP-matched control (Table 5) shows the gains do not require extra compute.

Significance. If the central claim holds under interface-controlled evaluation, the paper makes a useful and somewhat contrarian point for the merging literature: expert training is an orthogonal, under-explored design axis, and a trivially cheap first stage (prompt learning, 0.005% of parameters) materially changes the mergeability of the resulting task vectors. The finding that a frozen-backbone prompt baseline is competitive with SOTA merging at 20 tasks is independently valuable and likely to be cited on its own. The evaluation is broad (two backbones, 8/14/20 tasks, full FT and LoRA, five mergers, heterogeneous pools), the claims are falsifiable, and the needed control experiment is cheap. The mechanistic story (reduced representation drift, reduced update norms) is plausible and partially interface-independent, though not yet cleanly tied to the accuracy gains.

major comments (3)
  1. [§5.1, Table 1] Tables 1-2, §5.1: the headline comparison conflates two changes — expert training and the inference-time text interface. Merged DTEs are scored with task-specific learned prompts, merged Vision-FT with hand-crafted templates, so absolute accuracy gains mix 'better mergeable vision vectors' with 'better text interface.' The paper's own numbers show the interface alone carries most of the gap: on ViT-L/14, prompt learning with a frozen backbone (no merging at all) reaches 87.28/87.95/87.92 on 8/14/20 tasks — within 1-2 points of WA-merged DTEs (89.59/89.07/88.69) and 8-17 points above the WA-Vision-FT baseline (79.21/76.52/71.39) against which gains are measured. Claims like 'simple weight averaging of DTEs already outperforms Vision-FT merged by any model merging method' are therefore not established as merging claims. Two mitigations are available within the paper's own framework, and I
  2. [§4.3] Eq. (9)-(10), Fig. 5: the interpolation analysis, presented as mechanistic evidence for 'higher merge compatibility,' inherits the same interface confound. The text states that Vision-FT endpoints are evaluated with hand-crafted templates and DTE endpoints with learned prompts. Flatter mixture-loss profiles for DTEs may therefore partly reflect that the DTE encoders were co-adapted with their prompt interface, not intrinsically better linear mode connectivity of the vision parameters. The analysis should either use a matched interface across conditions or be explicitly caveated. Note that the other two mechanistic analyses are not confounded: representation drift (Fig. 4) and Frobenius update norms are vision-only measurements, and they do support the claim — the paper would be stronger leaning on these. Additionally, Fig. 5 shows only 'representative' task pairs; an aggregate statistic
  3. [§5.3, Fig. 6] Fig. 6: in the heterogeneous replacement experiment, the evaluation interface is swapped together with the expert: tasks whose expert was replaced by a DTE are evaluated with learned prompts, remaining tasks with templates. The monotone improvement as DTEs are added could therefore be driven substantially by the interface swap per replaced task rather than by improved cross-expert compatibility — the confound is exactly correlated with the x-axis. An interface-controlled variant (all tasks evaluated with templates, or all with learned prompts where feasible) is needed for the 'DTEs are beneficial even in mixed pools' conclusion. Also, only three runs are used and no variance band is shown; given that replacement order is random, error bars matter for the monotonicity claim.
minor comments (8)
  1. [Tables 1-2] Tables 1-2 report single-run numbers with no seed variance. Since expert training is stochastic, at least a subset (e.g., WA and one advanced merger on the 20-task benchmark) should report mean ± std over 2-3 seeds, or the paper should state that differences of the reported magnitude far exceed measured seed noise.
  2. [Fig. 10, §B.4] The relative-improvement annotations include +762%, +1809%, and +323%, implying near-zero Vision-FT baseline similarities on those tasks; absolute similarity values should be reported alongside. Two tasks (SUN397, STL10) show slightly negative changes (-3%, -1%) that go undiscussed. Also Fig. 4 reports ~64% relative improvement on ViT-L/14 while the norm analysis uses ViT-B/32 — the backbone for each analysis should be stated consistently in the main text.
  3. [§4.3, parameter-update analysis] The Frobenius-norm comparison (2.27±0.95 vs 2.58±0.81) has heavily overlapping distributions and a marginal paired t-test (p=0.025); the Wilcoxon result (p=0.001) is more convincing. Report effect size and consider a per-task scatter. A 12% norm reduction is modest relative to the ~64% representation-drift reduction, and the discrepancy deserves a sentence.
  4. [Table 2] LoRA-DTE separate experts are slightly worse than Vision-LoRA experts (87.73 vs 88.59 on 8 tasks) yet merge far better in normalized terms — this is actually one of the cleaner pieces of evidence for the mergeability claim (worse experts, better merges) and deserves explicit discussion rather than being left implicit.
  5. [§B.5, Table 4] Table 4: for SUN397 the remaining Stage-2 budget under the equal-budget protocol is only 0.046 PFLOPs, i.e., essentially no vision fine-tuning; clarify how Approach 2 handles such tasks and whether the SUN397 DTE in Table 5 is effectively prompt-only.
  6. [Table 5 caption, various] The caption of Table 5 says 'Subscripts report normalized accuracy' but the table uses brackets, not subscripts. Typos: 'setings' (§5.1); missing space in 'introduceDual-Tuned Experts' (§1 contributions); 'decompose' → 'decomposes' in §A.1 (Iso-C); inconsistent spacing in 'W A' throughout.
  7. [§5, 'Task-aware evaluation'] The task-aware evaluation assumption (task identity known at inference) is standard in this benchmark lineage, but the paper should state explicitly in the Limitations that reported numbers do not transfer to task-agnostic inference, since the learned prompts are per-task and no prompt-merging or task-inference mechanism is provided.
  8. [§B.2] Fig. 9's ablation is useful; it would help to state whether M=16 was also used for ViT-L/14 (d=768) and whether the saturation point shifts with backbone width.

Circularity Check

0 steps flagged · score 0.0 of 10

Empirical merging paper with no derivation-by-construction; claims are measured, not forced by inputs.

full rationale

This is a standard empirical methods paper: Dual-Tuned Experts are a two-stage training recipe (prompt learning, then vision fine-tuning), and the central claims are held-out multi-task accuracies under external merging operators (WA, TA, TIES, TSV-M, Iso-C) versus Vision-FT/LoRA baselines. Nothing in the chain defines the reported merged accuracy in terms of the training procedure or fits a parameter that is then relabeled as the target metric. Validation-chosen λ is ordinary hyperparameter selection. Representation-drift, Frobenius-norm, and linear-interpolation analyses in §4.3 are post-hoc geometric explanations, not inputs that algebraically force Table 1/2 numbers. Self-citations to overlapping-author merging work ([10], [24], [30]) supply baselines and tools, not uniqueness theorems or ansatze that make the DTE result true by construction. Evaluation confounds (learned prompts vs hand-crafted templates) are experimental-design concerns, not circularity. No step reduces a claimed prediction to its own definition or fit.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The claim rests on standard CLIP/contrastive classification, the empirical mode-connectivity folklore that justifies linear merging, the task-aware evaluation convention, and several training hyperparameters (context length, learning rates, merge scale λ). No new physical entities; DTE is a training protocol, not a postulated mechanism beyond smaller task vectors.

free parameters (4)
  • Number of prompt context tokens M = 16
    Set to 16 following CoOp default; ablation in supp. shows saturation near 16 but performance depends on this choice.
  • Prompt-stage learning rate = 1e-3
    Chosen as 1e-3 for Stage 1; not derived.
  • Merge scale λ (Task Arithmetic and related) = validation-best per setting
    Selected by linear search on held-out validation per standard protocol; directly affects reported merged accuracy for TA-family methods.
  • LoRA rank r = 16
    Fixed to 16 for LoRA experiments without rank sweep in the main claim.
assumptions (4)
  • domain assumption Linear combinations of task vectors in parameter space can yield usable multi-task models when experts stay near a shared pretrained basin (mode connectivity / flat minima).
    Invoked in §3.1 to justify arithmetic merging; taken from prior merging literature, not proved here.
  • domain assumption At inference, task identity is known so the matching prompt set or hand-crafted templates can be selected.
    Stated in §5 task-aware evaluation; required for both DTE and Vision-FT multi-task numbers.
  • domain assumption Unified-context CoOp prompts with frozen encoders are a valid Stage-1 semantic adaptation for CLIP classification.
    §3.3–4.2 adopt CoOp’s formulation as the prompt stage without re-deriving prompt learning theory.
  • standard math Standard arithmetic on floating-point weight tensors and cosine-similarity CLIP logits behave as in prior open implementations.
    Background for Eqs. (1)–(2) and all merge operators.
invented entities (1)
  • Dual-Tuned Expert (DTE)
    purpose: Name the two-stage expert (learned prompt matrix + task-specific vision encoder) claimed to be more merge-compatible than one-stage Vision-FT.
    Operational training protocol rather than a new latent variable; independent_evidence is the experimental merge gains and geometric measurements in this paper only.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking Expert Training for Model Merging with Prompt Learning." pith.science (2026). https://pith.science/paper/DMEKZWWH

@misc{pith2026260724465,
  author       = {Pith},
  title        = {Pith review of: Rethinking Expert Training for Model Merging with Prompt Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DMEKZWWH}},
  note         = {Machine review of arXiv:2607.24465}
}
read the original abstract

Model merging aims to combine multiple domain-specialized experts trained from a shared foundation model into a single multi-task model. Existing approaches largely focus on improving the merging procedure itself and typically assume experts obtained through full-parameter fine-tuning. In this work, we revisit expert training for model merging. We first show that prompt-based adaptation provides a strong baseline: independently learned prompts can be exploited across tasks while keeping the backbone fixed, avoiding the interference introduced by weight merging. Building on this observation, we introduce Dual-Tuned Experts (DTEs), a two-stage training strategy that first learns prompts and then fine-tunes the vision encoder. This reduces the magnitude of task-specific parameter updates and produces experts with higher merge compatibility. Experiments across multiple CLIP architectures, full fine-tuning, and LoRA experts show that DTEs consistently improve merged performance of standard merging approaches and remain effective even when combining heterogeneous sets of experts.

Figures

Figures reproduced from arXiv: 2607.24465 by the authors.

Figure 1
Figure 1. Per-task performance of CLIP ViT-L/14 across 8 tasks. Results include zero-shot performance (with and without tem￾plates), prompt learning, and Vision-FT. Prompt-tuned experts perform slightly worse than Vision-FT experts. 4. Dual-Tuned Experts for Model Merging 4.1. Prompt learning as a strong baseline for multi￾task inference We begin by examining the effectiveness of different adap￾tation strategies across a rang… view at source ↗
Figure 2
Figure 2. Average performance of merging CLIP ViT-L/14 across 8, 14, and 20 tasks. Results compare Task Arithmetic (TA), Iso￾C, and TSV-M against a prompt-learning baseline. The prompt￾learning baseline is highly competitive, especially as the number of tasks increases. This simple strategy already provides a strong baseline for multi-task inference. As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Method overview. (Left) Two-stage training of Dual-Tuned Experts (DTEs): first, prompt learning is performed with a frozen backbone; second, the vision backbone is fine-tuned while keeping learned prompts fixed. (Right) DTEs replace standard experts in model merging and can be combined with existing merging methods, after which standard CLIP classification is applied to the merged model. 0.0 0.2 0.4 0.6 0.8 1.0 Cosi… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Cosine similarity between the pre-trained model (CLIP ViT-L/14) and the DTEs and Vision-FT experts for the 8-Vision benchmark. DTEs exhibit consistently higher similarity to the pre￾trained model across all tasks, indicating smaller representation shifts after adaptati…
Figure 5
Figure 5. Figure 5: Loss along linear interpolations between pairs of task experts. Each curve interpolates between experts trained on tasks i and j. Flatter profiles indicate better compatibility for linear merging. along linear interpolations between vision encoders trained on different…
Figure 6
Figure 6. Figure 6: Performance as Vision-FT experts are progressively re￾placed by DTEs in a heterogeneous 20-task merge setting. DTE, we use the corresponding learned prompts; for tasks using Vision-FT experts, we use the provided hand-crafted templates. The results are shown in [PITH_…
Figure 7
Figure 7. Figure 7: Prompt Learning vs Vision Full Fine-Tuning performance on the 20-Vision benchmark. Results calculated for ViT-B/32 and using M = 16 context tokens for prompt learning. Cars DTD EuroSAT GTSRB MNIST RESISC45 SUN397 SVHN 20% 40% 60% 80% 8 Tasks Cars DTD EuroSAT MNIST GTSR…
Figure 8
Figure 8. Figure 8: Per-Task Accuracy (%) comparison of Vision-FT, WA (DTE), Iso-C (DTE) and WA (Vision-FT) for the 8,14 and 20-Vision benchmarks using ViT-L/14. ViT-B/32 and 768 for ViT-L/14). B.4. Merged Model Representation Drift We measure how much the vision representations change af…
Figure 9
Figure 9. Figure 9: Average Accuracy (%) on the 8-Vision benchmark as a function of number of context tokens for ViT-B/32 across different methods. We then measure the cosine similarity between the merged and individual adapted representations, averaged over the test set Di of task i: sim…
Figure 10
Figure 10. Figure 10: Cosine similarity between the individual experts and the WA-merged DTEs and Vision-FT for the 20-Vision benchmark and ViT-B/32. Merged DTEs exhibit consistently higher similarity to the individual experts across tasks, indicating higher represen￾tation preservation af…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 2 linked inside Pith

  1. [1]

    Food-101 – Mining Discriminative Components with Ran- dom Forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 – Mining Discriminative Components with Ran- dom Forests. InComputer Vision – ECCV 2014, pages 446– 461, Cham, 2014. Springer International Publishing. 11

    2014

  2. [2]

    Remote sens- ing image scene classification: Benchmark and state of the art.Proceedings of the IEEE, 2017

    Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sens- ing image scene classification: Benchmark and state of the art.Proceedings of the IEEE, 2017. 11

    2017

  3. [3]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. InProceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014. 11

    2014

  4. [4]

    Deep Learning for Classical Japanese Literature, 2018

    Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kita- moto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep Learning for Classical Japanese Literature, 2018. arXiv:1812.01718 [cs, stat]. 11

    arXiv 2018

  5. [5]

    An Analysis of Single-Layer Networks in Unsupervised Feature Learn- ing

    Adam Coates, Andrew Ng, and Honglak Lee. An Analysis of Single-Layer Networks in Unsupervised Feature Learn- ing. InProceedings of the Fourteenth International Confer- ence on Artificial Intelligence and Statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011. ISSN: 1938-7228. 11

    2011

  6. [6]

    EMNIST: Extending MNIST to handwritten let- ters

    Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andr ´e van Schaik. EMNIST: Extending MNIST to handwritten let- ters. In2017 International Joint Conference on Neural Net- works (IJCNN), pages 2921–2926, 2017. ISSN: 2161-4407. 11

    2017

  7. [7]

    Model breadcrumbs: Scaling multi-task model merging with sparse masks.Proceedings of the European Conference on Com- puter Vision, 2024

    Mohammad-Javad Davari and Eugene Belilovsky. Model breadcrumbs: Scaling multi-task model merging with sparse masks.Proceedings of the European Conference on Com- puter Vision, 2024. 2

    2024

  8. [8]

    Roy, and Michael Carbin

    Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy, and Michael Carbin. Linear mode connectivity and the lottery ticket hypothesis. InInternational Conference on Ma- chine Learning, 2020. 2

    2020

Show all 46 references
  1. [9]

    Clip with generative latent replay: a strong baseline for in- cremental learning.BMVC, 2024

    Emanuele Frascaroli, Aniello Panariello, Pietro Buzzega, Lorenzo Bonicelli, Angelo Porrello, and Simone Calderara. Clip with generative latent replay: a strong baseline for in- cremental learning.BMVC, 2024. 1

    2024

  2. [10]

    Task singular vectors: Reducing task in- terference in model merging.Proceedings of the IEEE con- ference on Computer Vision and Pattern Recognition, 2025

    Antonio Andrea Gargiulo, Donato Crisostomi, Maria Sofia Bucarelli, Simone Scardapane, Fabrizio Silvestri, and Emanuele Rodol`a. Task singular vectors: Reducing task in- terference in model merging.Proceedings of the IEEE con- ference on Computer Vision and Pattern Recognition,...

    2025

  3. [11]

    Challenges in representation learning: A re- port on three machine learning contests.Neural Networks, 64, 2013

    Ian Goodfellow, Dumitru Erhan, Pierre Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, Yingbo Zhou, Chetan Ramaiah, Fangxiang Feng, Ruifan Li, Xiaojie Wang, Dimitris Athanasakis, John Shawe-Taylor, Maxim Milakov, and...

    2013

  4. [12]

    EuroSAT: A Novel Dataset and Deep Learn- ing Benchmark for Land Use and Land Cover Classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. EuroSAT: A Novel Dataset and Deep Learn- ing Benchmark for Land Use and Land Cover Classification. IEEE Journal of Selected Topics in Applied Earth Observa- tions and Remote Sensing, 2019. 11

    2019

  5. [13]

    J. E. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.International Confer- ence on Learning Representations, 2021. 7

    2021

  6. [14]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco T ´ulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. InInternational Con- ference on Learning Representations, 2023. 1, 2, 6, 11

    2023

  7. [15]

    Vi- sual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. InEuropean conference on computer vision, pages 709–727. Springer, 2022. 2

    2022

  8. [16]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. InProceedings of the IEEE conference on Computer Vision and Pattern Recognition,

  9. [17]

    Self-regulating prompts: Foundational model adaptation without forgetting

    Muhammad Uzair Khattak, Syed Talal Wasim, Muzammal Naseer, Salman Khan, Ming-Hsuan Yang, and Fahad Shah- baz Khan. Self-regulating prompts: Foundational model adaptation without forgetting. InIEEE International Con- ference on Computer Vision, 2023. 2

    2023

  10. [18]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In IEEE International Conference on Computer Vision and Pat- tern Recognition Workshops, 2013. 11

    2013

  11. [19]

    Learning multiple layers of features from tiny images.University of Toronto,

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.University of Toronto,

  12. [20]

    Mnist hand- written digit database.ATT Labs [Online]., 2, 2010

    Yann LeCun, Corinna Cortes, and CJ Burges. Mnist hand- written digit database.ATT Labs [Online]., 2, 2010. 11

    2010

  13. [21]

    P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. InPro- ceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short P...

    2022

  14. [22]

    U-net transplant: the role of pre-training for model merging in 3d medical segmentation

    Luca Lumetti, Giacomo Capitani, Elisa Ficarra, Simone Calderara, Costantino Grana, Angelo Porrello, and Fed- erico Bolelli. U-net transplant: the role of pre-training for model merging in 3d medical segmentation. InInternational Conference on Medical Image Computing and Comput...

    2025

  15. [23]

    MAGMAX: leveraging model merg- ing for seamless continual learning

    Daniel Marczak, Bartlomiej Twardowski, Tomasz Trzcinski, and Sebastian Cygert. MAGMAX: leveraging model merg- ing for seamless continual learning. InProceedings of the European Conference on Computer Vision, 2024. 1

    2024

  16. [24]

    Bagdanov, and Joost van de Weijer

    Daniel Marczak, Simone Magistri, Sebastian Cygert, Bartłomiej Twardowski, Andrew D. Bagdanov, and Joost van de Weijer. No task left behind: Isotropic model merging with common and task-specific subspaces. InInternational Conference on Machine Learning, 2025. 1, 2, 4, 6, 11

    2025

  17. [25]

    Merging models with fisher-weighted averaging

    Michael Matena and Colin Raffel. Merging models with fisher-weighted averaging. InAdvances in Neural Informa- tion Processing Systems, 2021. 2

    2021

  18. [26]

    An empirical investigation of the role of pre- training in lifelong learning.Journal of Machine Learning Research, 24(214):1–50, 2023

    Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investigation of the role of pre- training in lifelong learning.Journal of Machine Learning Research, 24(214):1–50, 2023. 2

    2023

  19. [27]

    Linear mode connectivity in multitask and continual learning.arXiv preprint arXiv:2010.04495, 2020

    Seyed Iman Mirzadeh, Mehrdad Farajtabar, Dilan Gorur, Razvan Pascanu, and Hassan Ghasemzadeh. Linear mode connectivity in multitask and continual learning.arXiv preprint arXiv:2010.04495, 2020. 2

    2010 arXiv

  20. [28]

    Reading digits in natural images with unsupervised feature learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bis- sacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. InNeural Infor- mation Processing Systems Workshops, 2011. 11

    2011

  21. [29]

    Automated Flower Classification over a Large Number of Classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated Flower Classification over a Large Number of Classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pages 722–729, 2008. 11

    2008

  22. [30]

    Bag- danov, Simone Calderara, and Joost van de Weijer

    Aniello Panariello, Daniel Marczak, Simone Magistri, An- gelo Porrello, Bartłomiej Twardowski, Andrew D. Bag- danov, Simone Calderara, and Joost van de Weijer. Accurate and efficient low-rank model merging in core space. InAd- vances in Neural Information Processing Systems, 2...

    2025

  23. [31]

    Parkhi, Andrea Vedaldi, Andrew Zisserman, and C

    Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V . Jawahar. Cats and dogs. InIEEE Conference on Com- puter Vision and Pattern Recognition, 2012. 11

    2012

  24. [32]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InInternational Conf...

    2021

  25. [33]

    Transporting task vectors across different architectures without training.International Conference on Machine Learning, 2026

    Filippo Rinaldi, Aniello Panariello, Giacomo Salici, Angelo Porrello, and Simone Calderara. Transporting task vectors across different architectures without training.International Conference on Machine Learning, 2026. 1

    2026

  26. [34]

    Manning, A

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, A. Ng, and Christopher Potts. Re- cursive deep models for semantic compositionality over a sentiment treebank. InConference on Empirical Methods in Natural Language Processing, 2013. 11

    2013

  27. [35]

    Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition.Neural Net- works, 32:323–332, 2012. 11

    2012

  28. [36]

    Model merging with svd to tie the knots.International Conference on Learning Repre- sentations, 2025

    George Stoica, Pratik Ramesh, Boglarka Ecsedi, Leshem Choshen, and Judy Hoffman. Model merging with svd to tie the knots.International Conference on Learning Repre- sentations, 2025. 2, 7

    2025

  29. [37]

    Veeling, Jasper Linmans, Jim Winkens, Taco Co- hen, and Max Welling

    Bastiaan S. Veeling, Jasper Linmans, Jim Winkens, Taco Co- hen, and Max Welling. Rotation equivariant cnns for digi- tal pathology. InMedical Image Computing and Computer Assisted Intervention – MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018,...

    2018

  30. [38]

    Model soups: averaging weights of multi- ple fine-tuned models improves accuracy without increas- ing inference time

    Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Ko- rnblith, et al. Model soups: averaging weights of multi- ple fine-tuned models improves accuracy without increa...

    2022

  31. [39]

    Fashion- mnist: a novel image dataset for benchmarking machine learning algorithms, 2017

    Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion- mnist: a novel image dataset for benchmarking machine learning algorithms, 2017. 11

    2017

  32. [40]

    Sun database: Exploring a large col- lection of scene categories.International Journal of Com- puter Vision, 2016

    Jianxiong Xiao, Krista A Ehinger, James Hays, Antonio Tor- ralba, and Aude Oliva. Sun database: Exploring a large col- lection of scene categories.International Journal of Com- puter Vision, 2016. 11

    2016

  33. [41]

    TIES-merging: Resolving interference when merging models

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. TIES-merging: Resolving interference when merging models. InAdvances in Neural Information Processing Systems, 2023. 1, 2, 3, 4, 6, 11

    2023

  34. [42]

    Adamerging: Adap- tive model merging for multi-task learning

    Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. Adamerging: Adap- tive model merging for multi-task learning. InInternational Conference on Learning Representations, 2024. 2

    2024

  35. [43]

    Model merg- ing in llms, mllms, and beyond: Methods, theories, appli- cations, and opportunities.ACM Computing Surveys, 58(8): 1–41, 2026

    Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xi- aochun Cao, Jie Zhang, and Dacheng Tao. Model merg- ing in llms, mllms, and beyond: Methods, theories, appli- cations, and opportunities.ACM Computing Surveys, 58(8): 1–41, 2026. 1

    2026

  36. [44]

    Conditional prompt learning for vision-language mod- els

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language mod- els. InProceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2022. 2

    2022

  37. [45]

    Learning to prompt for vision-language models.Inter- national Journal of Computer Vision, 2022

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models.Inter- national Journal of Computer Vision, 2022. 2, 3, 6

    2022

  38. [46]

    De- mystifying mergeability: Interpretable properties to predict model merging success.International Conference on Ma- chine Learning, 2026

    Luca Zhou, Bo Zhao, Rose Yu, and Emanuele Rodol `a. De- mystifying mergeability: Interpretable properties to predict model merging success.International Conference on Ma- chine Learning, 2026. 6 A. Methods and Datasets A.1. Methods We evaluate our approach with several represe...

    2026

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.