Pith. sign in

REVIEW 3 major objections 4 minor 3 cited by

Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper frames multi-task model merging as a constrained optimization problem—keep the merged model as close as possible to each expert while preserving shared knowledge—and solves it with adaptive projective gradient descent.

desk verdict Solid empirical merging paper with a genuinely new method, but the data-free objective is unvalidated and the reported gains may owe more to the lambda heuristic than to the constrained optimization. read the letter →

arxiv 2501.01230 v3 pith:ZBCMBJ4X submitted 2025-01-02 cs.LG

Yongxian Wei , Anke Tang , Li Shen , Zixuan Hu , Chun Yuan , Xiaochun Cao This is my paper · ORCID
classification cs.LG
keywords modelmergingmulti-tasklearningtaskvectorsadaptiveprojectivegradientdescentdata-freeoptimizationsharedsubspacearithmeticLoRAfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Merging several fine-tuned expert models into one multi-task model usually means combining their task vectors, and existing methods resolve the resulting conflicts by dropping or orthogonalizing parameters. This paper argues that such pruning throws away task-specific information that is actually needed for performance. Instead, it proposes to minimize the gap between the merged model and each individual expert, subject to a constraint that shared knowledge is retained. The solution is an adaptive projective gradient descent (DOGE) that optimizes a data-free objective in the directions orthogonal to a shared subspace built from all tasks. If the approach holds, model merging can get close to true multi-task training without seeing any task data.

What carries the argument

The central object is the task vector τ_i = θ_i − θ_0, the difference between a fine-tuned expert and the pretrained model, interpreted as an accumulated gradient. The argument is carried by three ingredients: (i) a data-free objective that measures, for each task j, ⟨−τ_j, λ ∑_i (τ_i + Δ) − τ_j⟩², obtained by Taylor expansion and the NTK approximation ∇_θ L_j(θ_0) ≈ −τ_j / ∑_t α_t; (ii) a shared subspace S_share built by SVD of the task vectors (the first k left singular vectors of each, deduplicated by another SVD), onto which the gradient is projected so that updates happen only orthogonal to it, preserving shared knowledge; and (iii) a layer-wise merging coefficient λ_i^l = η/‖τ_i^l‖, which treats λ as an adaptive learning rate that balances task-vector magnitudes. The projection step is what converts conflict resolution into a constrained optimization problem rather than a pruning rule.

What would settle it

Fine-tune the same expert models with a much larger learning rate or many more steps so they leave the linear regime, then compute Eq. (5)'s objective with the true task data and compare it to the gradient proxy. If the proxy's error grows and DOGE's accuracy advantage over Task Arithmetic or Ties-Merging disappears, the central claim is falsified.

Watch

Extended reading notes

Core claim

The central claim is that task conflicts and shared knowledge should be handled separately: conflicts are resolved by optimizing a modification vector Δ added to each task vector, while shared knowledge is preserved by allowing gradient updates only in directions orthogonal to the subspace spanned by the task vectors' principal components. The paper derives a data-free objective by Taylor-expanding the loss gap between the merged model and the individual models around the pretrained weights, and approximating the pretrained gradient by the negative task vector scaled by the total learning-rate sum (an NTK/linear-regime assumption). Optimizing this objective with gradient descent whose steps are projected away from the shared subspace, plus a task-aware layer-wise merging coefficient λ_i^l = η/‖τ_i^l‖, yields the DOGE algorithm. In experiments across eight vision tasks (ViT-B/32 and ViT-L/14) and eight GLUE tasks (Flan-T5-base/large with LoRA), DOGE improves over Task Arithmetic by up to 11.6 percentage points and over AdaMerging by up to 5.8, and it also transfers to unseen tasks and corrupted test distributions.

Load-bearing premise

Everything rests on the assumption that fine-tuning stays in the linear (NTK) regime, so the task vector is exactly proportional to the negative gradient of the pretrained model; if experts are fine-tuned too far, the data-free objective no longer equals the true loss gap and the claimed gains may not persist.

Editorial extensions

If this is right

  • If the gap-minimization view is right, the merged model's performance on each task should approach the individual expert's performance, closing much of the gap that sparsification and orthogonality methods leave.
  • Data-free merging can compete with test-time adaptation methods, meaning no unlabeled test data is needed, which helps privacy and deployment.
  • The task-aware λ removes grid search over merging coefficients; one global η scales all layers and tasks.
  • DOGE can be applied on top of existing merging algorithms like Task Arithmetic and AdaMerging, acting as a plug-in rather than a replacement.
  • The method scales to more tasks (8→14→20) with relatively smaller degradation, suggesting it mitigates negative transfer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The NTK approximation used to derive the data-free objective may be the limiting factor; a testable extension is to measure how DOGE's gain degrades as fine-tuning moves away from the linear regime (larger learning rates, more steps, or non-LoRA full fine-tuning).
  • The shared subspace construction depends on SVD rank choice; a natural extension would be a data-dependent or task-similarity-weighted basis rather than uniform rank per task, especially for tasks with very different intrinsic dimensionality.
  • The same constrained-gap objective could be applied to heterogeneous models only if task vectors are first aligned to a common space, which the paper notes as a limitation.
  • DOGE's robustness on corrupted test sets suggests the projection orthogonal to the shared subspace may implicitly regularize the merged model; this could be tested by comparing its feature geometry against the individual experts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper frames multi-task model merging as a constrained optimization problem: minimize the gap between the merged model and each individually fine-tuned model, subject to a constraint that shared knowledge, represented by a shared subspace of task vectors, is preserved. The authors derive a data-free objective (Eq. (5)) via a first-order Taylor expansion and an NTK-style replacement of the pre-trained gradient by the task vector, then solve it by projected gradient descent with a task-aware, layer-wise merging coefficient. Experiments on CLIP ViT-B/32 and ViT-L/14 across 8, 14, and 20 vision tasks, on Flan-T5 with LoRA on GLUE, and on three 13B LLMs report consistent improvements over Task Arithmetic, Ties-Merging, AdaMerging, and other baselines, with ablations and robustness studies.

Significance. If the mechanism claimed in the paper were established, the work would be valuable: it offers a data-free, plug-and-play component that improves existing merging methods substantially (up to 11.6% over TA on the 8-task vision benchmark), with moderate computational overhead (121 s on ViT-B/32), and the ablations suggest each component contributes. The paper announces code availability and includes sensitivity analyses, robustness to corruptions, and generalization to unseen tasks. However, the central derivation rests on approximations that are not validated; until that is addressed, the theoretical framing remains a heuristic wrapper around an empirical recipe.

major comments (3)
  1. [Sec. 4.1, Eq. (5)] The data-free objective is obtained through two load-bearing approximations: the first-order Taylor expansion of L_j around θ0 and the NTK assumption that ∇L_j(θ0) ≈ -τ_j / Σ_t α_t. Neither approximation is validated in the paper, and the NTK replacement is additionally questionable because Adam-style fine-tuning does not make the task vector equal to the sum of raw gradients. The tension is concrete: for the near-orthogonal task vectors reported in Fig. 2 (cosine similarities mostly 0.01--0.15), the minimizer of Eq. (5) over λ (with Δ = 0) is close to 1, whereas App. B sets η so that the average λ is about 0.3. Since Δ is optimized with this fixed λ and is projected to stay orthogonal to the shared subspace, it cannot simply rescale the total vector to the proxy's preferred operating point. The paper therefore does not establish that the reported gains come from minimizing the gap in Eq. (3); they could come from the task-aware λ heuristic and the Δ perturbation alone. I ask for a direct empirical check that Eq. (5) tracks the true per-task loss gap (e.g., compute both on a small held-out subset before and after optimization), and a discussion of the λ discrepancy.
  2. [Sec. 4.2, Eq. (2) and Algorithm 1] The constraint S_share(θ*, θ0 + λ Σ τ_i) = 0 is never defined as a distance, so the reader cannot verify that the projection in Eq. (7) actually enforces it. More importantly, 'shared knowledge' is identified with the top singular vectors of the task vectors without justification; the claim that updates orthogonal to S_share 'retain shared knowledge' is an assertion, not a derived property. The ablations in Tab. 7 show empirical benefit, but they do not measure shared knowledge itself. Please provide a formal definition of S_share in Eq. (2) and, if possible, a quantitative proxy for shared-knowledge retention (e.g., performance on similar tasks or parameter overlap).
  3. [Sec. 5.1 / App. B] The top-30% magnitude mask from Ties-Merging is not present in the derivation of Eq. (5) or in Algorithm 1. This matters because Eq. (5) is an inner product of task vectors, and masking changes every inner product and every gradient. Please state where the mask is applied (before SVD, before Eq. (7), or only at the final output), or remove the mask from the method and show that the results still hold; as written, the optimized objective is not precisely specified.
minor comments (4)
  1. [App. D, Tab. 11] The claim that the approach 'consistently achieved the highest accuracy across most scenarios' is not fully supported by the table: under impulse noise, DOGE AM's average (65.1) is below AdaMerging's (68.0), and DOGE TA is below AdaMerging on every corruption; please qualify the claim.
  2. [Abstract] The abstract states 'Our code is available here' but the manuscript contains no URL; please provide the actual repository link.
  3. [Sec. 3.1] The decomposition τ_j = θ_T − θ_0 = −Σ_t α_t ∇_{θ_t} L_j(θ_t) is written as an equality, but with Adam-style fine-tuning the task vector is not exactly the sum of raw gradients; please present this as an approximation.
  4. [Sec. 5.1] The traditional MTL model is described as an 'upper bound', but in Tab. 1 it is 88.9 while individual models reach 90.8; consider describing it as a reference point or an upper bound for merging methods specifically.

Circularity Check

1 steps flagged · score 3.0 of 10

Mild circularity in the task-aware λ scale: η is chosen to reproduce Task Arithmetic's known 0.3 average λ, so part of the reported λ gain is inherited from the baseline; the central Δ/shared-subspace derivation is otherwise self-contained.

  1. fitted input called prediction [Sec. 4.3 Eq. (8); App. B; Abstract contribution claim (2.8% boost)]
    "λl i = η ||τ l i|| , ∀ l ≤ L ... This setting was chosen because the calculated average λ was close to 0.3, which is a beneficial scaling coefficient for the Task Arithmetic method, demonstrating that our approach is not tricky."

    The task-aware λ formula in Eq. (8) contains a free global magnitude η. Rather than deriving η from the stated data-free objective, Appendix B sets it by checking that the resulting average λ matches 0.3, a coefficient already known to work well for Task Arithmetic on the evaluated benchmarks. The abstract then credits 'simple task-aware λ' with a 2.8% performance boost. Thus the scale that makes the λ schedule useful is imported from the very baseline it is compared against; the per-task and per-layer normalization may still contribute independently, but the claimed training-free λ advantage is not fully derived from first principles and is partly a re-statement of the baseline's known optimal scale.

full rationale

DOGE's central derivation is not circular: Eq. (5) is an explicit data-free approximation of the loss-gap objective in Eq. (3), built from the task vectors and a stated NTK-based gradient substitution. The Taylor expansion and the gradient replacement are assumptions whose validity is not established, and the paper's chosen operating point (average λ≈0.3) is far from the proxy's own preferred λ≈1 for near-orthogonal task vectors; these are correctness and validity limitations, not circularity. No load-bearing uniqueness theorem or first-principles claim is imported from the authors' own prior work; the numerous self-citations are contextual and non-essential. The one genuine circular burden is the global scale η: Appendix B discloses that η was selected so the average λ matches 0.3, a scaling coefficient already known to help Task Arithmetic, while the abstract presents task-aware λ as a source of a 2.8% gain. This makes part of the λ contribution anchored to the baseline it is compared with. However, the main reported gains come from the Δ optimization and shared-subspace projection, which are evaluated by ablations and are independent of the λ scale choice, so the central algorithm does not reduce to this fitted scale.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or latent entities; the modification vector Delta and shared subspace S_share are optimization constructs within the existing parameter space. The main free parameters are eta and k, both set by heuristics rather than derived. The NTK and shared-subspace assumptions are load-bearing modeling choices.

free parameters (3)
  • Global scaling factor eta = 0.07 (vision), 0.15 (NLP)
    Chosen in App. B because the resulting average lambda is close to 0.3, a scaling known to work well for Task Arithmetic on the same benchmarks.
  • Subspace basis size k = rank of task vector divided by 8; shared subspace rank divided by 6
    Set by heuristic rule rather than derived; ablation in Fig. 4 shows sensitivity to rank, with optimal around 10-30% preserved rank.
  • Top-30% parameter retention = 30% of parameters with largest magnitudes
    Adopted from Ties-Merging in App. B without a dedicated ablation in this paper; it may affect the reported results.
assumptions (4)
  • domain assumption NTK linear-regime assumption: gradient of pre-trained loss is proportional to the negative task vector
    Invoked in Sec. 4.1 to replace nabla L_j(theta_0) with -tau_j / sum alpha_t, requiring fine-tuning to stay near theta_0.
  • standard math First-order Taylor expansion of the loss gap is valid
    Used in Eq. (4) to convert the loss gap into a linear inner-product objective; validity requires small parameter displacements.
  • domain assumption Task vectors represent accumulated gradients
    Assumes tau_j = theta_T - theta_0 = -sum alpha_t nabla L_t, standard for SGD fine-tuning, and that gradients at intermediate steps equal the gradient at theta_0.
  • ad hoc to paper Shared subspace built from singular vectors of task vectors captures shared knowledge
    The claim that the span of principal left singular vectors represents shared knowledge is assumed and validated only through ablations in Tabs. 6 and 7.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent." pith.science (2026). https://pith.science/paper/ZBCMBJ4X

@misc{pith2026250101230,
  author       = {Pith},
  title        = {Pith review of: Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZBCMBJ4X}},
  note         = {Machine review of arXiv:2501.01230}
}
abstract

Merging multiple expert models offers a promising approach for performing multi-task learning without accessing their original data. Existing methods attempt to alleviate task conflicts by sparsifying task vectors or promoting orthogonality among them. However, they overlook the fundamental target of model merging: the merged model performs as closely as possible to task-specific models on respective tasks. We find these methods inevitably discard task-specific information that, while causing conflicts, is crucial for performance. Based on our findings, we frame model merging as a constrained optimization problem ($\textit{i.e.}$, minimizing the gap between the merged model and individual models, subject to the constraint of retaining shared knowledge) and solve it via adaptive projective gradient descent. Specifically, we align the merged model with individual models by decomposing and reconstituting the loss function, alleviating conflicts through $\textit{data-free}$ optimization of task vectors. To retain shared knowledge, we optimize this objective by projecting gradients within a $\textit{shared subspace}$ spanning all tasks. Moreover, we view merging coefficients as adaptive learning rates and propose a task-aware, training-free strategy. Experiments show that our plug-and-play approach consistently outperforms previous methods, achieving state-of-the-art results across diverse architectures and tasks in both vision and NLP domains.

Figures

Figures reproduced from arXiv: 2501.01230 by the authors.

Figure 1
Figure 1. The effect of task numbers on average accuracy for ViT￾B/32, with error bars representing the 95% confidence interval. As the number of tasks increases, negative transfer becomes more pro￾nounced. Although our method initially performs lower than other methods, its performance decreases more slowly, demonstrating superior robustness when handling a larger number of tasks. SUN397Cars RESISC45 EuroSAT SVHN GTSRB MNIST… view at source ↗
Figure 2
Figure 2. (a) Cosine similarity matrices of task vectors for ViT￾B/32. (b) A schematic representation of the subspace spanned by the task representations, depicted as a two-dimensional plane. the merged model performs as closely as possible to task￾specific models on respective tasks. As shown in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An illustration of element magnitudes in the task vector, inspired by (Shen et al., 2024). Best viewed when zoomed in. shared subspace Sshare to preserve common representations (see [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The average accuracy changes corresponding to different rank ratios in the subspace under ViT-B/32 architecture. most vital, causing notable performance drops of 8.8% and 3.5% in vision tasks, and 2.0% and 0.9% in language tasks, respectively. With all modules included…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis

    cs.CV 2025-07 conditional novelty 6.0 of 10

    UniGlyph replaces pre-rendered glyph conditions with segmentation-derived masks in a ControlNet diffusion model, reporting gains on visual text rendering benchmarks.

  2. Weakly Supervised Temporal Action Localization via Dual-Prior Collaborative Learning Guided by Multimodal Large Language Models

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A training-time framework that uses MLLM-generated action descriptions to improve weakly supervised temporal action localization, with small mAP gains on THUMOS14 and ActivityNet-v1.2.

  3. Decoding the Alzheimer's Continuum: Interpretable Multi-Gate Routing for Diagnosis and Transition Prediction

    eess.IV 2025-08 reject novelty 5.0 of 10

    M3AD jointly classifies Alzheimer's stages and predicts cognitive transitions from T1-weighted MRI, reporting 95.13% diagnosis accuracy.

Reference graph

Works this paper leans on

17 extracted references · 7 canonical work pages · cited by 3 Pith papers

  1. [6]

    selfish” and “catastrophic

    Following Ties-Merging (Yadav et al., 2023), we retain only the top 30% of parameters with the largest magnitudes. We only apply our method to the linear layer in the model. For the implementation of our experiments, we employed PyTorch version 2.5 with Python 3.10. C. Compared Baselines Pre-trained: Uses a pre-trained model for each task without integrat...

    work page 2023

  2. [8]

    Fu- sionbench: A comprehensive benchmark of deep model fusion

    Tang, A., Shen, L., Luo, Y ., Hu, H., Du, B., and Tao, D. Fu- sionbench: A comprehensive benchmark of deep model fusion. arXiv preprint arXiv:2406.03280, 2024a. Tang, A., Shen, L., Luo, Y ., Yin, N., Zhang, L., and Tao, D. Merging multi-task models via weight-ensembling mixture of experts. In ICML, 2024b. Tang, A., Shen, L., Luo, Y ., Zhan, Y ., Hu, H., D...

    arXiv

  3. [11]

    Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities

    Yang, E., Shen, L., Guo, G., Wang, X., Cao, X., Zhang, J., and Tao, D. Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities. arXiv preprint arXiv:2408.07666, 2024a. Yang, E., Shen, L., Wang, Z., Guo, G., Chen, X., Wang, X., and Tao, D. Representation surgery for multi-task model merging. In ICML, 2024b. Yang, E., Wan...

    arXiv

  4. [12]

    Z., Albert, P., Rodriguez-Opazo, C., van den Hengel, A., and Abbasnejad, E

    Zhang, F. Z., Albert, P., Rodriguez-Opazo, C., van den Hengel, A., and Abbasnejad, E. Knowledge composition using task vectors with learned anisotropic scaling. In NeurIPS, 2024a. Zhang, Q., Liu, X., Li, W., Chen, H., Liu, J., Hu, J., Xiong, Z., Yuan, C., and Wang, Y . Distilling semantic priors from sam to efficient image restoration models. In CVPR, 202...

    arXiv

  5. [13]

    12 Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent A. Model Details For vision tasks, we employ pre-trained models from CLIP (Radford et al., 2021), fine-tuning them using the AdamW optimizer with a weight decay of 0.1 and a learning rate of 1 × 10−5. The downstream tasks encompass a variety of challenges. SUN397 (Xiao et al.,

    work page 2021

  6. [16]

    Method Cars EuroSAT RESISC45 GTSRB Avg

    Robustness to the test data distribution on ViT-B/32. Method Cars EuroSAT RESISC45 GTSRB Avg. Cars EuroSAT RESISC45 GTSRB Avg. Clean Test Set Corrupted Test Set (Motion Blur) Fisher Merging 66.0 92.7 83.7 78.7 80.3 60.7 57.6 81.7 78.4 69.6 Task Arithmetic 64.6 91.8 80.2 74.8 77.9 62.4 59.2 78.5 63.3 65.9 Ties-Merging 65.2 83.3 78.1 67.4 73.5 64.4 53.9 76....

    work page 2024

  7. [17]

    Method SUN397 Cars RESISC45 EuroSAT SVHN GTSRB MNIST DTD Avg

    Task-aware and training-free λ combined with Task Arithmetic. Method SUN397 Cars RESISC45 EuroSAT SVHN GTSRB MNIST DTD Avg. Task Arithmetic 55.2 54.9 66.7 78.9 80.2 69.7 97.3 50.4 69.1 + Task-wiseλ 61.4 62.5 70.0 82.8 71.3 66.4 95.1 56.1 70.7 + Layer-wiseλ 62.6 63.9 71.0 86.8 73.2 65.2 95.9 56.4 71.9 More task numbers. Tab. 14 illustrates the robustness o...

    work page 2024

  8. [1998]

    MAP: Low-compute model merging with amortized pareto fronts via quadratic approximation

    Li, L., Zhang, T., Bu, Z., Wang, S., He, H., Fu, J., Wu, Y ., Bian, J., Chen, Y ., and Bengio, Y . MAP: Low-compute model merging with amortized pareto fronts via quadratic approximation. In ICLR, 2025a. Li, W., Peng, Y ., Zhang, M., Ding, L., Hu, H., and Shen, L. Deep model fusion: A survey. arXiv preprint arXiv:2309.15698,

    arXiv

Show all 17 references
  1. [2014]

    We measure the models’ performance using top-1 accuracy as the primary metric (Horoi et al., 2024; Stoica et al., 2024; Wei et al., 2025)

    is a texture dataset with 5,640 images organized into 47 human-describable categories, designed for studying texture perception and classification. We measure the models’ performance using top-1 accuracy as the primary metric (Horoi et al., 2024; Stoica et al., 2024; Wei et al...

    2024

  2. [2016]

    Multi-task model merging via adaptive weight disentanglement

    11 Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent Xiong, F., Cheng, R., Chen, W., Zhang, Z., Guo, Y ., Yuan, C., and Xu, R. Multi-task model merging via adaptive weight disentanglement. arXiv preprint arXiv:2411.18729,

    arXiv

  3. [2017]

    Revisiting weight averaging for model merging

    Choi, J., Kim, D., Lee, C., and Hong, S. Revisiting weight averaging for model merging. arXiv preprint arXiv:2412.12153,

    arXiv

  4. [2019]

    Lines: Post-training layer scaling prevents forgetting and enhances model merging

    Wang, K., Dimitriadis, N., Favero, A., Ortiz-Jimenez, G., Fleuret, F., and Frossard, P. Lines: Post-training layer scaling prevents forgetting and enhances model merging. arXiv preprint arXiv:2410.17146, 2024a. Wang, K., Dimitriadis, N., Ortiz-Jimenez, G., Fleuret, F., and Fro...

    arXiv

  5. [2020]

    Concrete subspace learning based interference elimination for multi-task model fusion

    Tang, A., Shen, L., Luo, Y ., Ding, L., Hu, H., Du, B., and Tao, D. Concrete subspace learning based interference elimination for multi-task model fusion. arXiv preprint arXiv:2312.06173,

  6. [2021]

    Efficient and effec- tive weight-ensembling mixture of experts for multi-task model merging

    Shen, L., Tang, A., Yang, E., Guo, G., Luo, Y ., Zhang, L., Cao, X., Du, B., and Tao, D. Efficient and effec- tive weight-ensembling mixture of experts for multi-task model merging. arXiv preprint arXiv:2410.21804,

    arXiv

  7. [2023]

    From system 1 to system 2: A survey of reasoning large lan- guage models

    Li, Z.-Z., Zhang, D., Zhang, M.-L., Zhang, J., Liu, Z., Yao, Y ., Xu, H., Zheng, J., Wang, P.-J., Chen, X., et al. From system 1 to system 2: A survey of reasoning large lan- guage models. arXiv preprint arXiv:2502.17419, 2025b. Liu, J., Moreau, A., Preuss, M., Rapin, J., Rozi...

    arXiv

  8. [2024]

    M., M¨ollenhoff, T., Daheim, N., Gurevych, I., and Khan, M

    Maldonado, H. M., M¨ollenhoff, T., Daheim, N., Gurevych, I., and Khan, M. E. How to weight multitask finetuning? fast previews via bayesian model-merging.arXiv preprint arXiv:2412.08147,

    arXiv

  9. [2025]

    Task arithmetic in trust region: A training-free model merging approach to navigate knowledge conflicts

    Sun, W., Li, Q., Wang, W., Geng, Y .-a., and Li, B. Task arithmetic in trust region: A training-free model merging approach to navigate knowledge conflicts. arXiv preprint arXiv:2501.15065,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.