Pith. sign in

REVIEW 3 major objections 4 minor 45 references

Grounding Functional Similarity by Invariance-Aware Model Stitching

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Task-based model stitching can fabricate functional similarity; a new alignment objective, FuLA, avoids it.

desk verdict A plausible but not-yet-fully-supported new stitching objective that deserves a serious referee, provided the authors add a label-shuffled control and release code. read the letter →

arxiv 2505.20142 v2 pith:V5LBFQXV submitted 2025-05-26 cs.LG

Ioannis Athanasiadis , Anmar Karmush , Michael Felsberg This is my paper
classification cs.LG
keywords functionalsimilaritymodelstitchingLatentAlignmentrepresentationadversarialtrainingshortcutlearningcross-layerdirectmatching
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that standard model stitching, which measures functional similarity by how well one network's representations can be aligned to another's to solve a task, can report high similarity between models that actually rely on different information cues. The cause, the authors claim, is that task-based objectives let the stitching layer overfit to output-level cues, fabricating alignment that has nothing to do with shared internal function. They propose Functional Latent Alignment (FuLA), a stitching objective that trains the affine stitch to reproduce the end model's activations not only at the stitching layer but at every frozen layer after it, in a task-agnostic way. Across adversarial-training, shortcut, and cross-layer stitching experiments on ResNet variants, FuLA is reported to be less prone to these artifacts and to detect non-trivial alignments that direct matching misses. If correct, the paper overturns conclusions about functional similarity that were drawn from task-based stitching.

What carries the argument

The load-bearing object is the FuLA objective (Eq. 5): a weighted average of normalized Frobenius-distance "Hint" terms between the stitched model's activations and the end model's own activations, evaluated at the stitch layer and at each frozen layer after it. The first term coincides with direct matching (DM); the additional functional Hints force the alignment to be interpretable by the end model's later layers in the same way its native representations are. The stitching transformation is restricted to a $1\times1$ convolutional layer with bias, initialized by a Moore-Penrose solution of the DM objective, so any score reflects what a low-capacity map can preserve, not what a high-capacity adapter can force.

What would settle it

Run FuLA and task-based stitching on a pair of models that are behaviorally identical on the test distribution but whose internal representations past the stitch layer are related by a nonlinear but invertible transform; FuLA should score them dissimilar, and if independent behavioral probes judge them functionally interchangeable, FuLA's criterion is too strict.

Watch

Extended reading notes

Core claim

The central claim is that functional similarity between two networks should be judged by forward-backward compatibility: a low-capacity affine map must be able to send the front model's representation into the end model's layers such that those layers, frozen, produce the same internal activations they produce on their own native input. Under this criterion, the paper defines a stitched model $h = g_{>j} \circ T_\theta \circ f_{\le i}$ and optimizes $T_\theta$ to minimize normalized Frobenius distances between stitched and native activations at the stitch layer and at every subsequent layer of the end model (Eq. 5). The authors show that the previously dominant task-loss matching (TLM) and soft-label matching (SLM) objectives can fabricate functional alignment, for example by reporting non-robust accuracy when stitching between non-robust models, while FuLA interpolates functional properties smoothly and reveals discrepancies, such as a sharp drop in functional similarity at the penultimate layer when morphing non-robust to robust models, that direct matching misses.

Load-bearing premise

The paper's conclusions rest on defining functional similarity as the ability of a low-capacity affine map to reproduce the end model's internal activations through its frozen layers, and if that definition is not what functional similarity should mean, the experimental results do not establish it.

Editorial extensions

If this is right

  • Conclusions about functional similarity drawn from task-based stitching, for instance that robust and non-robust models are functionally similar with respect to clean accuracy, need to be re-examined under alignment-based objectives.
  • FuLA and Hint-based stitching should be preferred over TLM and SLM when the goal is to know whether two models share internal functional properties rather than whether they can both solve the task.
  • Directionality in cross-layer stitching is an intrinsic property of functional similarity, not an artifact of the task loss: earlier layers can be guided by deeper layers, but deeper guidance does not help shallow input layers.
  • Functional discrepancies between models can be localized in depth, because FuLA's per-layer Hints can pinpoint where similarity breaks down, such as the penultimate layer, earlier than direct matching.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If FuLA's notion is adopted, previously published stitching-based similarity scores between robust and non-robust models are best interpreted as upper bounds contaminated by task cues until re-run under FuLA.
  • The same forward-backward compatibility criterion could serve as a training signal or selection criterion for model merging and reassembly, where a stitch must transfer representations across independently trained components.
  • A natural stress test would be to construct two models with identical input-output behavior but a known nonlinear reparametrization of internal layers; FuLA should report low similarity, revealing that it measures representational compatibility rather than purely behavioral equivalence.
  • The shortcut experiments suggest that FuLA's advantage grows under distribution shift between training and test, so evaluating it on out-of-distribution benchmarks for language or multimodal models is a direct next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Functional Latent Alignment (FuLA), a new optimality condition for model stitching that aligns the front model's representations not only at the stitching layer but also through the frozen layers of the end model (Eq. 5). The authors argue that standard task-based stitching (TLM and SLM) can fabricate functional alignment by overfitting to task-specific cues, while FuLA is a more reliable, task-agnostic alternative. The method is evaluated on adversarial training, shortcut learning, cross-layer stitching, and class-subset stitching experiments using CIFAR-10 and a low-resolution ImageNet-10 benchmark with ResNet variants. The central claims are that task-based stitching can hallucinate non-existent functional alignment and that FuLA recovers non-trivial alignments missed by direct matching.

Significance. If the central claims hold, the paper would substantially challenge previous conclusions about functional similarity between robust and non-robust models and would provide a practical alternative to task-based stitching. The method is clearly specified, the experimental suite is extensive across architectures and settings, and the paper honestly acknowledges the lack of ground truth for functional similarity. However, the evidence for FuLA's superior reliability is currently qualitative and lacks the controls and statistical support needed to make the claims load-bearing. The paper is a useful contribution to the ongoing debate on model stitching, but its main assertions require stronger empirical grounding before they can be accepted as established.

major comments (3)
  1. [Section 3.1, Fig. 4] The load-bearing claim that TLM/SLM 'fabricate' or 'hallucinate' functional alignment under adversarial training is not supported by a control experiment. The paper attributes the non-zero robust accuracy achieved when stitching two non-robust models with α=1 to overfitting on task cues, but it does not test whether the effect survives label shuffling or random-label stitching. Without such a control, the observed behavior could be a genuine effect of the stitching layer learning robustness from the adversarial examples used during stitching, rather than evidence of task-cue exploitation. This distinction is central to the paper's main conclusion, so the authors should either add the control or substantially soften the 'fabrication' claim.
  2. [Eq. (5) and Section 5 (Limitations)] FuLA's objective is, by construction, the criterion used to define functional similarity, and the validation then checks whether FuLA matches intuitive expectations in designed scenarios. The paper's own Limitations section concedes that 'definitive conclusions remain out of reach due to the lack of ground truth.' To make the 'more reliable metric' claim convincing, the authors need to specify falsifiable predictions or quantitative criteria for reliability, such as consistency across seeds, area-under-stitching-curve comparisons, or agreement with a clearly stated operational definition of functional similarity that is not identical to the FuLA objective.
  3. [Section 3.1 and Figures 4–6] The experimental evidence for FuLA's superior stability and 'non-existent functional similarity' at the penultimate layer is presented only through visual inspection of stitching plots without error bars, confidence intervals, or statistical tests. In particular, the statement that FuLA reports 'random chance performance' at the penultimate layer is not quantified against a defined chance level. The authors should provide repeated runs or confidence intervals and, where possible, a quantitative measure of the differences between stitching settings so that the reader can assess whether the claimed differences are meaningful.
minor comments (4)
  1. [Eq. (5)] The indexing in Eq. (5) is ambiguous: the sum over l runs to k, but the text states that FuLA aligns representations only up to the penultimate layer. Please clarify whether the output layer is included and align the notation with the definition of k-layer networks.
  2. [Section 5] There is a typo in the Limitations paragraph: 'lack to ground truth' should be 'lack of ground truth.'
  3. [Abstract and Introduction] The statement that 'recent studies' highlight the limitation of compatible representations misleadingly similar cites Smith et al. 2025, but the reference list gives 'Smith and Marcu'; please ensure the citation format is consistent.
  4. [Experimental Setting] The paper does not state the number of random seeds or whether the reported stitching plots are from a single run. Reporting variability would greatly improve the interpretability of the qualitative comparisons.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor definitional aspect: FuLA's conclusions about functional alignment partly restate its own objective, but the central comparison against external baselines is independent.

  1. self definitional [Section 2.3, Eq. (5) and Section 3.3, Fig. 11 discussion]
    "we have observed a smooth transition as we included deeper functional terms (i.e., through weighting the FuLA objective the Eq. (5)) as shown in Fig. 11. Based on that, we conclude that earlier layers can retain functionally relevant information when guided by deeper layers."

    The 'functionally relevant information' is defined by the functional-Hint terms (Ll_Hint, l ≥ j+1) of Eq. (5), the very terms whose weights are changed. Observing a smooth transition as deeper terms are added, and then concluding that earlier layers are 'guided by deeper layers,' restates the construction of the FuLA objective rather than an independent empirical finding. This is not load-bearing for the paper's main claim that task-based stitching can fabricate alignment, because that claim is evaluated against external baselines on the AT and shortcut testbeds; it is a mild interpretive circularity in the cross-layer analysis.

full rationale

This is primarily a method-proposal paper. FuLA's objective (Eq. 5) is normative: it defines functional similarity as the degree to which low-capacity affine stitching can reproduce the end model's internal activations through frozen layers. The paper openly acknowledges that 'definitive conclusions remain out of reach due to the lack to ground truth' (Section 5). The central comparisons are external: TLM, SLM, DM/Hint are evaluated on the same AT, shortcut, and cross-layer testbeds, and the claims about task-based stitching 'fabricating alignment' are empirical interpretations rather than fitted parameters renamed as predictions. There are no load-bearing self-citations; the references to knowledge distillation and FitNets are used as inspiration, not as authority that forces the result. The only noticeable circularity is the definitional one: when the paper concludes that FuLA 'captures functional alignment' or that deeper layers 'guide' earlier layers, it is partly restating that FuLA optimizes exactly those deeper-layer alignment terms. This is a minor, acknowledged limitation, not a hidden reduction of the central result, so the overall circularity score is low.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The central method depends on a normative definition of functional similarity, a capacity constraint on the stitch layer, and acceptance of constructed testbeds as proxies for ground truth. The only tunable ingredients are the Hint weights and the choice of which layers to align.

free parameters (2)
  • Hint weights C = uniform default
    Relative weighting of structural vs functional hints in Eq. (5); a design choice that controls the metric. Fig. 11 varies the number of functional hints, showing sensitivity.
  • Number of functional hints = all layers between stitching and penultimate
    Choosing which layers to align is a modeling decision; results in Fig. 6 and Fig. 11 depend on it.
assumptions (3)
  • domain assumption Functional similarity is appropriately defined by alignment of internal latent activations through frozen layer compositions of the end model
    Eq. (5) operationalizes similarity as feature distance; this normative premise is assumed, not derived.
  • domain assumption Affine 1x1 convolutions are a sufficiently constrained family that the stitching layer cannot fabricate alignment
    Section 2.3: 'the family of transformations... must be of low capacity'; relied on to interpret stitch accuracy as similarity.
  • domain assumption Ground truth for functional similarity is unavailable, so constructed testbeds are assumed to reflect true similarity
    Section 5 Limitations concedes definitive conclusions out of reach; evaluation relies on expected behavior in AT/shortcut settings.
invented entities (1)
  • Forward-backward compatibility requirement
    purpose: Presented as the basis of invariance-aware stitching in the abstract; not formalized in the full text
    No definition or equation appears; if this is central to the claimed contribution, its absence is a gap.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Grounding Functional Similarity by Invariance-Aware Model Stitching." pith.science (2026). https://pith.science/paper/V5LBFQXV

@misc{pith2026250520142,
  author       = {Pith},
  title        = {Pith review of: Grounding Functional Similarity by Invariance-Aware Model Stitching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V5LBFQXV}},
  note         = {Machine review of arXiv:2505.20142}
}
read the original abstract

In deep learning, functional similarity evaluation quantifies the extent to which independently trained models learn similar input--output relationships. In model stitching, functional similarity is framed as representation forward compatibility, i.e., whether the representations of two models can be aligned to solve a given task. Recent studies, however, highlight a critical limitation: models relying on different information cues can still produce compatible representations, making them appear misleadingly similar (Smith et al., 2025). We attribute this failure to standard model stitching being inherently blind to the invariance properties of the stitched models. To address this limitation, we introduce the forward--backward compatibility requirement under which we formulate the invariance-aware model stitching. Through analyzing key stitching configurations, we study the interplay between forward and backward compatibility, showing that invariance-aware model stitching provides a more principled approach to functional similarity evaluation while revealing functional discrepancies previously obscured.

Figures

Figures reproduced from arXiv: 2505.20142 by the authors.

Figure 1
Figure 1. Conceptual visualization for different degrees of functional alignment. The [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The proposed model stitching by FuLA in relation to other model stitching settings. In the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. DM in practice. Implementing DM as Hint: To better understand how different optimization settings in model stitching impact functional similarity evaluation, we focus on the relevant aspects of this comparison. Based on our analysis, we found that DM, as proposed by Balogh and Jelasity [2], may be unfairly underperforming compared to SLM and TLM due to two key factors: (i) using fewer samples to learn Tθ and (ii) no… view at source ↗
Figures from the paper (27 more)
Figure 4
Figure 4. Figure 4: Cross-task stitching under AT – ResNet18. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Same-task stitching under AT (α = 1) – ResNet18 Model stitching with AT poses particular interest, as distinct behaviors emerge across different stitching configurations (e.g., first column in [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Adversarial example penulti￾mate layer similarity. In particular, the penultimate layers exhibit a decaying trend in functional similarity for the non-robust to robust stitching configuration, starting at 6th layer, whereas the others remain relatively steady throughou…
Figure 7
Figure 7. Figure 7: Pattern-based vs location-based shortcuts. Next, we explore a setting where shortcuts[16], previously unseen by both the front and end models, are introduced during model stitching. We adapt the AT stitching setup for shortcut learning using models trained on clean dat…
Figure 8
Figure 8. Figure 8: Stitching under shortcuts. In [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Model stitching when DM￾initialized and trained on a subset of classes. Severe discrepancies between shortcut and clean data in￾dicate overfitting to the training distribution, in this case, the shortcuts. Based on this, it is evident that task-based model stitching is…
Figure 10
Figure 10. Figure 10: Cross-layer stitching between regularly trained models. Rows and columns represent the [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Cross￾layer stitching with FuLA when includ￾ing up to three func￾tional Hints. We observe that model stitching by Hint (i.e., realizing the DM) also exhibits directional behavior (e.g., [5,3] is higher than [3,5] in self-stitching in [PITH_FULL_IMAGE:figures/full_fig…
Figure 12
Figure 12. Figure 12: Visualizing shortcut configurations. Cross-Layer Stitching: During cross-layer stitching, we train the affine transformation using the same training configuration as used for the corresponding layer stitching. We performed stitching for all 9 × 9 layer combinations. W…
Figure 13
Figure 13. Figure 13: Cross-task stitching under AT – ResNet18 (w/o residuals). [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Same-task stitching under AT – ResNet18 (w/o residuals). [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: Cross-task stitching under AT – ResNet18. [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: Same-task stitching under AT – ResNet18. [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]
Figure 17
Figure 17. Figure 17: Cross-task stitching under AT – ResNet34. [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]
Figure 18
Figure 18. Figure 18: Same-task stitching under AT – ResNet34. [PITH_FULL_IMAGE:figures/full_fig_p017_18.png]
Figure 19
Figure 19. Figure 19: Cross-task stitching under shortcuts – ResNet18 (w/o residuals). [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: Same-task stitching under shortcuts – ResNet18 (w/o residuals). [PITH_FULL_IMAGE:figures/full_fig_p018_20.png]
Figure 21
Figure 21. Figure 21: Cross-task stitching under shortcuts – ResNet18. [PITH_FULL_IMAGE:figures/full_fig_p018_21.png]
Figure 22
Figure 22. Figure 22: Same-task stitching under shortcuts – ResNet18. [PITH_FULL_IMAGE:figures/full_fig_p019_22.png]
Figure 23
Figure 23. Figure 23: Cross-task stitching under shortcuts – ResNet34. [PITH_FULL_IMAGE:figures/full_fig_p019_23.png]
Figure 24
Figure 24. Figure 24: Same-task stitching under shortcuts – ResNet34. [PITH_FULL_IMAGE:figures/full_fig_p019_24.png]
Figure 25
Figure 25. Figure 25: Stitching on a subset of classes – ResNet18 (w/o residuals). [PITH_FULL_IMAGE:figures/full_fig_p020_25.png]
Figure 26
Figure 26. Figure 26: Stitching on a subset of classes – ResNet18. [PITH_FULL_IMAGE:figures/full_fig_p020_26.png]
Figure 27
Figure 27. Figure 27: Stitching on a subset of classes – ResNet34. [PITH_FULL_IMAGE:figures/full_fig_p020_27.png]
Figure 28
Figure 28. Figure 28: Cross-layer stitching – ResNet18 (w/o residuals). [PITH_FULL_IMAGE:figures/full_fig_p021_28.png]
Figure 29
Figure 29. Figure 29: Cross-layer stitching – ResNet18 [PITH_FULL_IMAGE:figures/full_fig_p022_29.png]
Figure 30
Figure 30. Figure 30: Cross-layer stitching – ResNet34. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_30.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 25 canonical work pages

  1. [1]

    On the functional similarity of robust and non-robust neural representations

    András Balogh and Márk Jelasity. On the functional similarity of robust and non-robust neural representations. In International Conference on Machine Learning, pages 1614–1635. PMLR, 2023

    work page 2023

  2. [2]

    How not to stitch representations to measure similarity: Task loss matching versus direct matching

    András Balogh and Márk Jelasity. How not to stitch representations to measure similarity: Task loss matching versus direct matching. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 15472–15480, 2025

    work page 2025

  3. [3]

    Revisiting model stitching to compare neural representations

    Yamini Bansal, Preetum Nakkiran, and Boaz Barak. Revisiting model stitching to compare neural representations. Advances in neural information processing systems, 34:225–236, 2021

    work page 2021

  4. [4]

    Representation learning: A review and new perspectives

    Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. arxiv 2012. arXiv preprint arXiv:1206.5538, 2012

    arXiv 2012

  5. [5]

    From bricks to bridges: Product of invariances to enhance latent space communication

    Irene Cannistraci, Luca Moschella, Marco Fumero, Valentino Maiorca, and Emanuele Rodolà. From bricks to bridges: Product of invariances to enhance latent space communication. arXiv preprint arXiv:2310.01211, 2023

    arXiv 2023

  6. [6]

    Cross-layer distillation with semantic calibration

    Defang Chen, Jian-Ping Mei, Yuan Zhang, Can Wang, Zhe Wang, Yan Feng, and Chun Chen. Cross-layer distillation with semantic calibration. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 7028–7036, 2021

    work page 2021

  7. [7]

    What do cnns learn in the first layer and why? a linear systems perspective

    Rhea Chowers and Yair Weiss. What do cnns learn in the first layer and why? a linear systems perspective. In International Conference on Machine Learning , pages 6115–6139. PMLR, 2023

    work page 2023

  8. [8]

    Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

    Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pages 2206–2216. PMLR, 2020

    2020

Show all 45 references
  1. [9]

    Similarity and matching of neural network representations

    Adrián Csiszárik, Péter K ˝orösi-Szabó, Akos Matszangosz, Gergely Papp, and Dániel Varga. Similarity and matching of neural network representations. Advances in Neural Information Processing Systems, 34:5656–5668, 2021

    2021

  2. [10]

    Reliability of cka as a similarity measure in deep learning

    MohammadReza Davari, Stefan Horoi, Amine Natik, Guillaume Lajoie, Guy Wolf, and Eu- gene Belilovsky. Reliability of cka as a similarity measure in deep learning. arXiv preprint arXiv:2210.16156, 2022

    2022 arXiv

  3. [11]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  4. [12]

    Grounding representation similarity through statistical testing

    Frances Ding, Jean-Stanislas Denain, and Jacob Steinhardt. Grounding representation similarity through statistical testing. Advances in Neural Information Processing Systems, 34:1556–1568, 2021

    2021

  5. [13]

    Hierarchical nucleation in deep neural networks

    Diego Doimo, Aldo Glielmo, Alessio Ansuini, and Alessandro Laio. Hierarchical nucleation in deep neural networks. Advances in Neural Information Processing Systems, 33:7526–7536, 2020

    2020

  6. [14]

    Dermatologist-level classification of skin cancer with deep neural networks

    Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. Dermatologist-level classification of skin cancer with deep neural networks. nature, 542(7639):115–118, 2017

    2017

  7. [15]

    Automatic generation and detection of highly reliable fiducial markers under occlusion

    Sergio Garrido-Jurado, Rafael Muñoz-Salinas, Francisco José Madrid-Cuevas, and Manuel Jesús Marín-Jiménez. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition, 47(6):2280–2292, 2014

    2014

  8. [16]

    Shortcut learning in deep neural networks

    Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020. 10

    2020

  9. [17]

    Explaining and harnessing adversar- ial examples

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.6572, 2014

    2014 arXiv

  10. [18]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  11. [19]

    Lu, and Marin Soljacic

    Adriano Hernandez, Rumen Dangovski, Peter Y . Lu, and Marin Soljacic. Model stitching: Looking for functional similarity between representations. In SVRHM 2022 Workshop @ NeurIPS, 2022. URL https://openreview.net/forum?id=7bHLCO5FQdB

    2022

  12. [20]

    Show, attend and distill: Knowledge distillation via attention-based feature matching

    Mingi Ji, Byeongho Heo, and Sungrae Park. Show, attend and distill: Knowledge distillation via attention-based feature matching. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 7945–7952, 2021

    2021

  13. [21]

    If you’ve trained one you’ve trained them all: inter-architecture similarity increases with robustness

    Haydn T Jones, Jacob M Springer, Garrett T Kenyon, and Juston S Moore. If you’ve trained one you’ve trained them all: inter-architecture similarity increases with robustness. In Uncertainty in Artificial Intelligence, pages 928–937. PMLR, 2022

    2022

  14. [22]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  15. [23]

    Similarity of neural network representations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In International conference on machine learning , pages 3519–3529. PMLR, 2019

    2019

  16. [24]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, 2017

    2017

  17. [25]

    Understanding image representations by measuring their equivariance and equivalence

    Karel Lenc and Andrea Vedaldi. Understanding image representations by measuring their equivariance and equivalence. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 991–999, 2015

    2015

  18. [26]

    Subspace adversarial training

    Tao Li, Yingwen Wu, Sizhe Chen, Kun Fang, and Xiaolin Huang. Subspace adversarial training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13409–13418, 2022

    2022

  19. [27]

    Function-consistent feature distillation

    Dongyang Liu, Meina Kan, Shiguang Shan, and Xilin Chen. Function-consistent feature distillation. arXiv preprint arXiv:2304.11832, 2023

    2023 arXiv

  20. [28]

    Exploring inter-channel correlation for diversity-preserved knowledge distillation

    Li Liu, Qingle Huang, Sihao Lin, Hongwei Xie, Bing Wang, Xiaojun Chang, and Xiaodan Liang. Exploring inter-channel correlation for diversity-preserved knowledge distillation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8271–8280, 2021

    2021

  21. [29]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017

    2017 arXiv

  22. [30]

    The tunnel effect: Building data representations in deep neural networks

    Wojciech Masarczyk, Mateusz Ostaszewski, Ehsan Imani, Razvan Pascanu, Piotr Miło´s, and Tomasz Trzcinski. The tunnel effect: Building data representations in deep neural networks. Advances in Neural Information Processing Systems, 36:76772–76805, 2023

    2023

  23. [31]

    Efficient estimation of word representations in vector space

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013

    2013 arXiv

  24. [32]

    Linear mode connectivity in multitask and continual learning

    Seyed Iman Mirzadeh, Mehrdad Farajtabar, Dilan Gorur, Razvan Pascanu, and Hassan Ghasemzadeh. Linear mode connectivity in multitask and continual learning. arXiv preprint arXiv:2010.04495, 2020

    2010 arXiv

  25. [33]

    Insights on representational similarity in neural networks with canonical correlation

    Ari Morcos, Maithra Raghu, and Samy Bengio. Insights on representational similarity in neural networks with canonical correlation. Advances in neural information processing systems, 31, 2018. 11

    2018

  26. [34]

    Relative representations enable zero-shot latent space communication

    Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communication. arXiv preprint arXiv:2209.15430, 2022

    2022 arXiv

  27. [35]

    Measuring representational robustness of neural networks through shared invariances

    Vedant Nanda, Till Speicher, Camila Kolling, John P Dickerson, Krishna Gummadi, and Adrian Weller. Measuring representational robustness of neural networks through shared invariances. In International Conference on Machine Learning, pages 16368–16382. PMLR, 2022

    2022

  28. [36]

    Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth

    Thao Nguyen, Maithra Raghu, and Simon Kornblith. Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth. arXiv preprint arXiv:2010.15327, 2020

    2010 arXiv

  29. [37]

    Stitchable neural networks

    Zizheng Pan, Jianfei Cai, and Bohan Zhuang. Stitchable neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16102–16112, 2023

    2023

  30. [38]

    Stitched vits are flexible vision backbones

    Zizheng Pan, Jing Liu, Haoyu He, Jianfei Cai, and Bohan Zhuang. Stitched vits are flexible vision backbones. In European Conference on Computer Vision, pages 258–274. Springer, 2024

    2024

  31. [39]

    Fitnets: Hints for thin deep nets

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014

    2014 arXiv

  32. [40]

    Comparing apples and oranges: is stitching similarity a load of spheres? In NeurIPS 2024 Workshop on Scientific Methods for Understanding Deep Learning, 2024

    Damian Smith and Antonia Marcu. Comparing apples and oranges: is stitching similarity a load of spheres? In NeurIPS 2024 Workshop on Scientific Methods for Understanding Deep Learning, 2024

    2024

  33. [41]

    Robustness may be at odds with accuracy

    Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint arXiv:1805.12152, 2018

    2018 arXiv

  34. [42]

    Knowledge distillation via softmax regression representation learning

    Jing Yang, Brais Martinez, Adrian Bulat, and Georgios Tzimiropoulos. Knowledge distillation via softmax regression representation learning. In International Conference on Learning Representations, 2021

    2021

  35. [43]

    Deep model reassembly

    Xingyi Yang, Daquan Zhou, Songhua Liu, Jingwen Ye, and Xinchao Wang. Deep model reassembly. Advances in neural information processing systems, 35:25739–25753, 2022

    2022

  36. [44]

    Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer

    Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928, 2016

    2016 arXiv

  37. [45]

    Formation of representations in neural networks

    Liu Ziyin, Isaac Chuang, Tomer Galanti, and Tomaso Poggio. Formation of representations in neural networks. arXiv preprint arXiv:2410.03006, 2024. 12 A Experimental Setup In this section, we cover the implementation details needed for reproducing our results. A.1 Baseline Trai...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.