Pith. sign in

REVIEW 4 major objections 6 minor 28 references

Model Fusion via Retrofitting

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Retrofitting fuses independently trained models by fitting their levels to weighted centroids of base-model neurons.

desk verdict A genuinely useful two-stage fusion recipe with solid zero-shot empirical wins, but the advertised approximation guarantee is for a slightly different algorithm than the one actually run. read the letter →

arxiv 2507.00037 v2 pith:6TNMQRRA submitted 2025-06-18 cs.LG cs.AI

Phoomraphee Luenam , Andreas Spanopoulos , Amit Sant , Thomas Hofmann , Sotiris Anagnostidis , Sidak Pal Singh This is my paper · ORCID
classification cs.LGcs.AI
keywords modelfusionneuronalignmentrepresentationmatchingimportancescoresnon-IIDdatazero-shotK-meansclusteringretrofitting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that model fusion—combining several independently trained neural networks into one usable model without retraining—works much better when the fusion is framed as a two-stage representation-matching problem: first cluster the parent networks' neurons into groups, then adjust the fused network's weights so its own neurons reproduce the importance-weighted centroids of those clusters. The authors report that this approach outperforms existing zero-shot fusion methods, especially when parents were trained on non-overlapping or highly skewed data, and in several cases approaches the accuracy of running all parents as an ensemble. A sympathetic reader would care because the method requires no data sharing, no retraining of the parents, and no architecture-specific alignment tricks, which makes it a candidate tool for federated and private-data settings where models trained on separated clients must be combined.

What carries the argument

The central object is the representation-cost decomposition of Equation (2): for a fixed level, the total weighted squared distance between base-model neurons and their nearest fused-model neuron splits into an approximation error term and a grouping error term, with a target vector T_k defined as the importance-weighted centroid of the base neurons clustered to fused neuron k. This identity is what lets the algorithm decouple the intractable joint optimization into a clustering step (minimize grouping error) and a weight-fitting step (minimize approximation error), with Theorem 2 providing an optimality guarantee for Hungarian matching and a (9+epsilon)-approximation for K-means fusion under linear-level assumptions.

What would settle it

If a reader fused two or more models under non-IID sharding and found that the best fused model, with the same fusion data and no fine-tuning, performed at or below the accuracy of the best single parent model while OTFusion with fine-tuning exceeded it, the central claim of zero-shot advantage would fail in that regime.

Watch

Extended reading notes

Core claim

The paper introduces a neuron-centric fusion framework in which a neural network is decomposed into sequential 'levels' (for example, individual layers, transformer blocks, or the classifier head). For each level, the authors define a representation cost: every neuron of every base model is matched to the closest neuron of the fused model, with the squared distance weighted by a neuron importance score. They prove that this cost decomposes exactly into a grouping error (how well parent neurons cluster around importance-weighted centroids) plus an approximation error (how well the fused model's neurons reproduce those centroids), and they show that for linear levels, minimizing the decoupled objective reduces to a weighted least-squares fit, while the clustering step is solved either by Hungarian matching (equal-size models) or by importance-weighted K-means (general widths and more than two models). The central empirical claim is that this two-stage 'retrofitting' procedure, with no fine-tuning, consistently beats OTFusion, Git Re-Basin, ZipIt!, and other baselines on non-IID, sharded, and full-dataset benchmarks, with the largest gains in zero-shot non-IID settings where prior methods often collapse to near-random accuracy.

Load-bearing premise

The whole construction assumes that clustering the parent models' neurons based only on their activations gives assignments that stay close to optimal once the fused model's weights are actually fitted, and the theory only bounds the decoupled objective with that fixed assignment rather than the true end-to-end objective.

Editorial extensions

If this is right

  • If the central claim holds, zero-shot fusion becomes practical in federated and privacy-preserving settings: models trained on disjoint client data can be merged into a single model without sharing data or running iterative federated-training rounds.
  • Fusing models of different widths and depths becomes routine, since the K-means variant does not require matched layer sizes and empirically handles width mismatches better than optimal-transport baselines.
  • The same framework acts as a data-efficient model-compression tool, distilling a larger teacher into a smaller student using only teacher activations and a small fusion set, outperforming standard knowledge distillation under limited data.
  • Because the method works for any architecture that can be decomposed into sequential levels, it offers a unified recipe for VGGs, ResNets, and Transformers, removing the need for architecture-specific alignment implementations.
  • In full-dataset settings, the fused model can be fine-tuned to exceed individual base-model accuracy while retaining single-model inference cost, closing much of the gap to ensembles.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The success of the method suggests that neuron-level clustering of activations is a more transferable correspondence across heterogeneous models than permutation alignment, because it allows many-to-one and many-to-many matching rather than one-to-one neuron swaps.
  • The importance-weighting component is likely to matter most precisely when models disagree strongly, since attribution scores computed on each model's own local data can suppress noisy or irrelevant neurons from the clustering step.
  • A natural testable extension is to apply the same centroid-retrofitting objective to large language models, where levels are transformer blocks and the fusion dataset could be a small public corpus, potentially providing a data-efficient alternative to weight interpolation.
  • If the representation-cost identity is as robust as claimed, it also suggests a general principle for cross-task merging: the fused model should be defined by reproducing the salient intermediate statistics of its parents, rather than by averaging the parents' parameters.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces a family of neuron-centric model fusion algorithms, Hungarian Fusion (HF) and K-means Fusion (KF), which operate in two stages per network level: first, base-model neurons are grouped (by one-to-one Hungarian matching for equal-size models, or by importance-weighted K-means otherwise) and importance-weighted centroids are formed; second, the fused model's level weights are fit to these centroids by closed-form least squares or gradient descent. The authors provide a decomposition of an L2 representation cost into grouping and approximation errors (Theorem 1) and state approximation guarantees for linear levels (Theorem 2). Experiments cover VGG, ResNet, and ViT architectures under sharded, non-IID, and full-dataset regimes, with a central claim of consistent zero-shot improvements over OTFusion, Git Re-Basin, ZipIt!, KD, and LP, at times approaching ensemble accuracy. The paper additionally contributes an open-source reimplementation of baseline fusion algorithms.

Significance. If the claims hold, the paper would offer a practical, architecture-flexible zero-shot fusion method for heterogeneous and privacy-constrained settings, and the representation-cost decomposition is an elegant way to separate clustering from function approximation. The empirical scope is unusually broad, spanning multiple architectures, data partitions, and fusion dataset sizes, and the open-source code with reimplemented baselines is a genuine service to the community. The importance-score mechanism is a useful extension over activation-matching baselines. However, the theoretical guarantees as written do not cover the algorithm that is actually run, and one headline baseline comparison uses unequal fusion-data budgets; these issues must be resolved before the central claims are fully supported.

major comments (4)
  1. [§4.1, Algorithm 1 and Eq. (1)-(3)] Algorithm 1 fixes the neuron-to-cluster assignment R_k (line 4) using only base-model activations, before the fused-level weights w are optimized (line 9). The representation cost in Eq. (1), however, takes a minimum over fused neurons with respect to the final fused output zF; after fitting w, the nearest fused neuron for a base neuron can differ from its assigned cluster. The algorithm therefore minimizes the surrogate in Eq. (3), not the objective in Eq. (1). Theorem 1's decomposition is stated for R_k defined as the minimum-cost assignment for the final w, so it does not justify the fixed assignment used by Algorithm 1. The paper should either prove a bound on the gap between the surrogate and Eq. (1), or explicitly present the method as optimizing a decoupled surrogate and restrict the theoretical claims accordingly.
  2. [§B.1/B.2 and Theorem 2(b)] Theorem 2(b) states a (9+epsilon)-approximation guarantee for KF 'when using the local-search algorithm of Kanungo et al. (2002)', and the proof requires projecting activations onto the column space of the preceding level's outputs. The implementations described in Section B.1 use Lloyd's algorithm with k-means++ for the general grouping case, and Section B.2 states that the gradient variants do not project activations before clustering. Thus the stated guarantee does not apply to the algorithm evaluated in the experiments. Please either implement the Kanungo local-search clustering and the projection for the variants whose performance is reported, or qualify the theorem so that it is not presented as a property of the implemented KF methods.
  3. [§A, proof of Theorem 2(a)] The proof of Theorem 2(a) asserts that 'minimizing the total cost reduces to minimizing the grouping error.' This step is not justified: for a fixed matching R, the approximation error term in Eq. (2) depends on the centroids T_k, which are determined by which neurons are matched. A matching with smaller grouping error can produce centroids that are harder to approximate by the fused linear layer, so the matching that minimizes grouping error need not minimize the sum of grouping and approximation error. The claimed optimality of Hungarian Fusion for the decoupled objective is therefore not established. Please provide a condition under which the approximation error is independent of the matching (e.g., all centroids lie in the column space of the previous layer's outputs for every matching), or prove the joint optimality directly.
  4. [Table 17 / Table 2] The main ViT sharded comparison uses 5000 fusion samples for KF-Gradient and HF-Gradient, while Transformer OTFusion uses only 200 samples (footnote in Table 17). Since fusion dataset size materially affects performance (see Table 7 of the same paper), the headline zero-shot advantage over Transformer OTFusion in Table 2 may be partly a data-budget artifact rather than an algorithmic improvement. The paper should either run Transformer OTFusion with the same 5000 samples, or provide a sensitivity analysis at matched sample counts to show the conclusion is unchanged. This is load-bearing because the sharded ViT results are a central piece of the empirical claim.
minor comments (6)
  1. [§4, text after Eq. (3)] The sentence 'One may either keep the weights of keep the weights of previous levels frozen' contains a duplicated phrase; it should read 'keep the weights of previous levels frozen'.
  2. [§F.2] The sentence 'In practice we as we primarily tested on like models, there were obvious answers that we used' is grammatically broken; please rephrase.
  3. [§F.5] The text contains a corrupted character sequence 'â ˘AˇT' in the sentence about constant-factor slowdown; it should be an em dash or comma.
  4. [§A, Theorem 1 proof] In the final displayed equation of the proof, the second term inside the brackets should be `(T_k - z_j)^2`, not `s_j (T_k - z_j)^2`; the extra `s_j` is a typo.
  5. [Tables 18 and 20] The 4-way full-dataset results in Table 18 are from a single seed, and the Tiny-ImageNet 2-way results in Table 20 are averaged over 2 seeds; please state the resulting uncertainty explicitly or provide additional seeds for these tables.
  6. [§8 and §C] The conclusion claims that the algorithms are 'the first to successfully incorporate neuron importance scores in model fusion,' but Section C acknowledges that OTFusion already uses importance scores as the probability measure in its optimal transport formulation; please soften the claim or clarify the specific form of novelty.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem 1 is an algebraic identity, Algorithm 1 optimizes its own decoupled objective, benchmarks are external, and self-citations are baselines rather than load-bearing.

full rationale

The paper's central derivation is self-contained. Theorem 1 (Appendix A) is an algebraic identity: the representation cost J_w is decomposed into approximation and grouping errors by adding and subtracting the importance-weighted cluster mean T_k, and the cross-term vanishes exactly because T_k is defined as the weighted mean of the assigned outputs. No quantity in Eq. (2) is defined in terms of the result it is used to establish. Algorithm 1 explicitly optimizes the decoupled objective of Eq. (3) for fixed clusters R_k, and the theoretical claims in Theorem 2 are stated for that decoupled objective: part (a) is an exact optimality result for the linear-sum assignment problem solved by the Hungarian algorithm, and part (b) follows from the Kanungo et al. (9+epsilon) approximation for weighted k-means combined with an orthogonal-decomposition lower bound in the proof. The proof is arithmetically explicit and does not assume the conclusion. A scope mismatch exists between the (9+epsilon) guarantee, which requires Kanungo local search, and the implementation using Lloyd's algorithm with k-means++ (Section B.1); this is a correctness/applicability gap, not circular reasoning. Self-citations, including Theus et al. (2026) for LMC evidence and Imfeld et al. (2024) and Singh and Jaggi (2020) as comparison baselines, are used for motivation or as methods to beat, not as load-bearing justification for the proposed algorithm's correctness. Empirical claims are benchmarked on external datasets (CIFAR-10, CIFAR-100, Tiny-ImageNet, ImageNet-1k, BloodMNIST) against external methods (OTFusion, Git Re-Basin, ZipIt!, FedAvg, FedMA, KD, LP), and no fitted parameter is renamed as a prediction; final accuracies are measured on held-out test data. The paper therefore does not exhibit circularity under any of the enumerated patterns.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method rests on activation-distance matching inherited from OTFusion/Git Re-Basin; on a DAG level-decomposition claim; on decoupling the matching from weight fitting; and on attribution scores computed on train data. Hyperparameters such as lambda, learning rate, and fusion dataset size are hand-chosen. No new physical entities are introduced.

free parameters (6)
  • L2 regularization lambda for HF-Linear = 1e-3
    Section F.1: found to work well across experiments; chosen by hand, not derived.
  • L2 regularization lambda for KF-Linear = 10
    Section F.1: chosen by hand for KF-Linear.
  • Gradient variant hyperparameters = lr=1e-3, weight decay=1e-4, epochs=100, batch size=32, val split=0.1, perturbation epsilon=1.0
    Table 9: single config that generalizes; hand-chosen.
  • Fusion dataset size per experiment = 400 (VGG), 5000 (ViT), 25k (ImageNet-1k)
    Chosen per experiment; KF-Gradient improves monotonically with fusion data size (Table 4), so reported numbers depend on this choice.
  • Number of clusters (K) = fused layer width
    Set to desired fused width; for HF one-to-one matching, no choice; for KF, K is a hyperparameter.
  • Neuron importance scoring method = Uniform / Conductance / DeepLIFT
    Treated as a hyperparameter (Section F.4.3); best results often use Conductance or DeepLIFT.
assumptions (5)
  • domain assumption Activation-based squared L2 distance is the correct neuron-closeness metric.
    Inherited from OTFusion and Git Re-Basin; this choice drives Eq. (1) and the entire method.
  • domain assumption A DNN with skip connections can be decomposed into a DAG of sequential levels such that each level is a function of the previous level's output.
    Section 3: branching layers (skip connections, transformer blocks) are contained in a single level; this is an architectural claim that must hold for the composition to be valid.
  • ad hoc to paper The assignment of base neurons to fused neurons can be fixed using base activations alone (decoupling).
    Section 4 and Algorithm 1: clusters R_k are computed from base activations before fused weights are known; the true argmin in Eq. (1) depends on fused outputs. Theorem 2 only bounds the decoupled objective.
  • standard math Kanungo et al. (2002) local search gives a (9+epsilon)-approximation for weighted k-means.
    Invoked in Theorem 2(b) proof; external result.
  • domain assumption Neuron importance scores computed on training data extend to the fusion (skewed) distribution.
    Section F.4.2: scores computed per model on its own data; used to weight centroids at fusion time.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Model Fusion via Retrofitting." pith.science (2026). https://pith.science/paper/6TNMQRRA

@misc{pith2026250700037,
  author       = {Pith},
  title        = {Pith review of: Model Fusion via Retrofitting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6TNMQRRA}},
  note         = {Machine review of arXiv:2507.00037}
}
read the original abstract

Model fusion seeks to combine independently trained neural networks into a single model without retraining, but is complicated by representational divergence arising from permutation invariance, random initialization, and heterogeneous training data. Existing methods struggle particularly in zero-shot settings under non-IID data distributions, and are often limited to specific architectures or pairwise fusion. We introduce a neuron-centric family of fusion algorithms that frames fusion as a principled representation-matching problem: intermediate neurons across parent models are grouped into target representations, which the fused model's corresponding sub-networks are then trained to approximate. Unlike prior work, our approach incorporates neuron attribution scores to bias alignment toward salient features, and can be applied to any architecture modularizable as a DAG of levels -- empirically validated on VGGs, ResNets, and ViTs. Experiments across standard benchmarks show consistent improvements over existing fusion methods, with the largest gains in zero-shot and non-IID scenarios. Code is available at https://github.com/AndrewSpano/model-fusion-via-retrofitting.

Figures

Figures reproduced from arXiv: 2507.00037 by the authors.

Figure 1
Figure 1. Overview of our method. Given two MLPs with two hidden layers of size 4, we compute the layer activations, cluster their neurons, and train the first layer of the fused model to match the centroids with the MSE loss. The process is repeated in subsequent levels. Algorithm 1 Model Fusion via Activation Retrofitting Require: Base models M = {Mk} n k=1, neuron impor￾tance scores {s Mk,l j } n k=1 L l=1, fusion dataset … view at source ↗
Figure 2
Figure 2. ViT Landscapes for CIFAR10, showing the fused model from KF-Gradient using DeepLIFT scores. Loss values were clipped to a maximum of 4.6 to keep a smooth gradient. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Effect of neuron importance scores on clustering. Preactivations and importance-weighted K-means clusterings of neurons from two VGG11 models trained on non-IID splits of CIFAR-10 at the third convolutional layer (256 channels per model, K=256). Neuron preactivations for 200 samples are projected to 2D via PCA. Black dots denote the densest cluster, gray dots de￾note singleton clusters, and other colors denote remai… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Histogram of Conductance and DeepLIFT Importance Scores for the Last Convolutional Layer of a VGG11 on one non-IID split (split by two) on CIFAR-100. For the general case, where a level is a non-linear differentiable function of its weights, a local minimum can be obta…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 10 canonical work pages

  1. [1]

    Git re-basin: Merging models modulo permutation symmetries

    Samuel K Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging models modulo permutation symmetries. arXiv preprint arXiv:2209.04836, 2022

    arXiv 2022

  2. [2]

    Np-hardness of euclidean sum-of-squares clustering

    Daniel Aloise, Amit Deshpande, Pierre Hansen, and Preyas Popat. Np-hardness of euclidean sum-of-squares clustering. Machine learning, 75: 0 245--248, 2009

    work page 2009

  3. [3]

    Berthold and Frank H \" o ppner

    Michael R. Berthold and Frank H \" o ppner. On clustering time series using euclidean distance and pearson correlation. CoRR, abs/1601.02213, 2016. URL http://arxiv.org/abs/1601.02213

    arXiv 2016

  4. [4]

    Convergence properties of the k-means algorithms

    Leon Bottou and Yoshua Bengio. Convergence properties of the k-means algorithms. Advances in neural information processing systems, 7, 1994

    work page 1994

  5. [5]

    Autoaugment: Learning augmentation policies from data

    Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501, 2018

    arXiv 2018

  6. [6]

    How important is a neuron? arXiv preprint arXiv:1805.12233, 2018

    Kedar Dhamdhere, Mukund Sundararajan, and Qiqi Yan. How important is a neuron? arXiv preprint arXiv:1805.12233, 2018

    arXiv 2018

  7. [7]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  8. [8]

    Correlation-based pruning algorithm with weight compensation for feedforward neural networks

    Shaimaa EK Ebid, Samah El-Tantawy, Doaa Shawky, and Hany L Abdel-Malek. Correlation-based pruning algorithm with weight compensation for feedforward neural networks. Neural Computing and Applications, pages 1--17, 2025

    work page 2025

Show all 28 references
  1. [9]

    The pile: An 800gb dataset of diverse text for language modeling

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020

    2020 arXiv

  2. [10]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  3. [11]

    Flat minima

    Sepp Hochreiter and J \"u rgen Schmidhuber. Flat minima. Neural computation, 9 0 (1): 0 1--42, 1997

    1997

  4. [12]

    Transformer fusion with optimal transport

    Moritz Imfeld, Jacopo Graldi, Marco Giordano, Thomas Hofmann, Sotiris Anagnostidis, and Sidak Pal Singh. Transformer fusion with optimal transport. arXiv preprint arXiv:2310.05719, 2023

    2023 arXiv

  5. [13]

    A local search approximation algorithm for k-means clustering

    Tapas Kanungo, David M Mount, Nathan S Netanyahu, Christine D Piatko, Ruth Silverman, and Angela Y Wu. A local search approximation algorithm for k-means clustering. In Proceedings of the eighteenth annual symposium on Computational geometry, pages 10--18, 2002

    2002

  6. [14]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  7. [15]

    Captum: A unified and generic model interpretability library for pytorch, 2020

    Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz-Richardson. Captum: A unified and generic model interpretability library for pytorch, 2020. URL h...

    2020 arXiv

  8. [16]

    The hungarian method for the assignment problem

    Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2 0 (1-2): 0 83--97, 1955

    1955

  9. [17]

    On the surprising effectiveness of attention transfer for vision transformers

    Alex Li, Yuandong Tian, Beidi Chen, Deepak Pathak, and Xinlei Chen. On the surprising effectiveness of attention transfer for vision transformers. Advances in Neural Information Processing Systems, 37: 0 113963--113990, 2024

    2024

  10. [18]

    Least squares quantization in pcm

    Stuart Lloyd. Least squares quantization in pcm. IEEE transactions on information theory, 28 0 (2): 0 129--137, 1982

    1982

  11. [19]

    Communication-efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273--1282. PMLR, 2017

    2017

  12. [20]

    Vit-cifar

    omihub777. Vit-cifar. https://github.com/omihub777/ViT-CIFAR. Accessed: 2025-05-16

    2025

  13. [21]

    Compute trends across three eras of machine learning

    Jaime Sevilla, Lennart Heim, Anson Ho, Tamay Besiroglu, Marius Hobbhahn, and Pablo Villalobos. Compute trends across three eras of machine learning. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1--8. IEEE, 2022

    2022

  14. [22]

    Learning important features through propagating activation differences

    Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference on machine learning, pages 3145--3153. PMlR, 2017

    2017

  15. [23]

    Model fusion via optimal transport

    Sidak Pal Singh and Martin Jaggi. Model fusion via optimal transport. Advances in Neural Information Processing Systems, 33: 0 22045--22055, 2020

    2020

  16. [24]

    Non-iid data in federated learning: A survey with taxonomy, metrics, methods, frameworks and future directions

    David Solans, Mikko Heikkila, Andrea Vitaletti, Nicolas Kourtellis, Aris Anagnostopoulos, Ioannis Chatzigiannakis, et al. Non-iid data in federated learning: A survey with taxonomy, metrics, methods, frameworks and future directions. arXiv preprint arXiv:2411.12377, 2024

    2024 arXiv

  17. [25]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319--3328. PMLR, 2017

    2017

  18. [26]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  19. [27]

    Federated learning with matched averaging

    Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Federated learning with matched averaging. arXiv preprint arXiv:2002.06440, 2020

    2002 arXiv

  20. [28]

    Source prompt: Coordinated pre-training of language models on diverse corpora from multiple sources

    Yipei Xu, Dakuan Lu, Jiaqing Liang, Jin Zhao, Xintao Wang, Hengkui Wu, Ken Chen, Liujiang Liu, Yingsi Xin, Xuepeng Liu, et al. Source prompt: Coordinated pre-training of language models on diverse corpora from multiple sources. In Proceedings of the 33rd ACM International Conf...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.