Pith. sign in

REVIEW 3 major objections 6 minor 56 references

ReStNet: A Reusable & Stitchable Network for Dynamic Adaptation on IoT Devices

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ReStNet stitches two pretrained models at their most CKA-similar layers and trains only the connector, matching or beating the larger parent across five benchmarks.

desk verdict A reasonable CKA-based stitching heuristic undercut by a frozen-head protocol that makes the Transformer experiments impossible as written. read the letter →

arxiv 2506.09066 v1 pith:N3CZV42J submitted 2025-06-08 cs.CV cs.AI

Maoyu Wang , Yao Lu , Jiaqi Nie , Zeyu Wang , Yun Lin , Qi Xuan , Guan Gui This is my paper
classification cs.CVcs.AI
keywords modelstitchingCenteredKernelAlignmentCKAIoTdeploymentpretrainedmodelsrepresentationsimilaritydynamicresourceadaptationhomogeneousandheterogeneous
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes ReStNet, a way to reuse two publicly available pretrained models by cutting one after its early layers and appending the later layers of the other, with a single trainable stitching layer in between. It claims that choosing the seam by Centered Kernel Alignment (CKA), taking the layer pair whose representations are most similar, avoids the prohibitive cost of searching all possible stitching points, and that fine-tuning only that connector is enough to reach accuracy close to or above the larger parent model. The motivation is IoT deployment: a single stitched model can be reconfigured under different parameter budgets, unlike a compressed model whose architecture is fixed after pruning or quantization. Across CNN-CNN, Transformer-Transformer, and CNN-Transformer combinations on five datasets, the paper reports flexible accuracy-efficiency trade-offs at a fraction of the training cost of training a model from scratch.

What carries the argument

The central mechanism is model stitching guided by Centered Kernel Alignment (CKA), a normalized measure of how similar two networks' activation patterns are on the same inputs. For every pair of layers, one from the front model and one from the behind model, ReStNet computes the normalized Hilbert-Schmidt Independence Criterion between the two networks' activations on a shared batch, builds a similarity matrix, and greedily takes the highest-scoring pair that satisfies the parameter budget. The assembly then keeps the early layers of the larger model, appends the deeper layers of the smaller model, and inserts a lightweight stitching layer (a 1D convolution, linear layer, or 2D convolution plus flatten, depending on the architecture pair) that adapts the feature dimensions at the seam. Only this connector is updated during training; everything else stays frozen.

What would settle it

Record exactly which weights change during the DTD and Pets runs. If the final classification layer is replaced or retrained for 37 or 40 classes, then "only the stitching layer is fine-tuned" is false; if the layer truly stays fixed at 1,000 classes, the cross-entropy loss against 37- or 40-class labels cannot be computed, so some head change must have occurred. Observing which one happens settles whether the reported cost claim holds as stated.

Watch

Extended reading notes

Core claim

The central discovery is that a stitched network can reach competitive task accuracy with remarkably few trainable parameters when the seam is chosen by representation similarity: in the reported experiments the connector accounts for as little as 0.07M trainable parameters, and the hybrid can match or slightly exceed the larger parent (for example, Swin-B followed by DeiT-S surpasses Swin-B on DTD, and ResNet18 followed by Swin-Ti surpasses Swin-Ti on ImageNette). The paper's explanation is that layers with high CKA similarity already compute related representations, so a thin trainable bridge between them is sufficient to align feature spaces, and that Slow-to-Fast stitching (large model's early layers followed by small model's deeper layers) is consistently better than the reverse direction at equal or smaller parameter counts.

Load-bearing premise

The load-bearing premise is that freezing both parent models and training only the connecting layer is a complete recipe for adapting to a new task, yet the paper does not say how a frozen output layer built for 1,000 ImageNet classes is reconciled with target datasets that have 10, 37, or 40 classes.

Editorial extensions

If this is right

  • A single ReStNet can be re-sliced at different CKA-chosen seams to hit different parameter budgets, so one deployment pipeline can serve devices with different memory and compute constraints.
  • Because only the stitching layer is trained, adapting to a new dataset costs far less than training or fully fine-tuning a model; reported trainable parameters are as low as 0.07M in the CNN-CNN case.
  • CKA similarity is a usable predictor of stitching success: the paper reports a consistent positive correlation between seam similarity and downstream accuracy across CNN and Transformer pairs.
  • Slow-to-Fast stitching (large model front, small model back) dominates Fast-to-Slow in the reported experiments, beating it in accuracy while using fewer total parameters.
  • Heterogeneous stitching (CNN front, Transformer back) works as well as homogeneous stitching, suggesting that cross-family reuse of pretrained models is feasible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the CKA-seam rule could serve as a general ranking criterion for reusing any two pretrained models, and it is worth testing whether the same correlation with transferability holds for speech or text encoders.
  • A testable extension is to run the same seam search with measured on-device latency or energy as the resource constraint instead of parameter count, since a CNN feeding a Transformer can have a very different latency profile than its parameter count suggests.
  • The paper's Slow-to-Fast result suggests a design principle for hybrid networks: put the high-capacity, high-level-feature layers early and the lightweight classifier-adjacent layers late, which invites experiments on whether the order preference persists across different task families.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. ReStNet constructs a hybrid network from two pre-trained models by keeping the early layers of a larger model and the later layers of a smaller one, connected by a trainable stitching layer. The stitch point is selected as the layer pair with maximal CKA similarity subject to a parameter budget, and only the stitching layer is fine-tuned. Experiments are reported for CNN-CNN stitched models on CIFAR-10/CIFAR-100, Transformer-Transformer on DTD/Pets, and CNN-Transformer on ImageNette, with claims of flexible accuracy-parameter trade-offs at very low training cost.

Significance. The problem addressed—dynamic deployment across heterogeneous IoT devices by reusing pre-trained models—is timely, and the paper's decomposition into stitch-point selection, stitching direction, and fine-tuning strategy is useful. The pseudo-code in Algorithms 1 and 2, the four-way fine-tuning comparison in Table IV, and the inclusion of heterogeneous CNN-Transformer stitching are assets. However, the central quantitative claim is not supported as written: for the Transformer experiments the 'only the stitching layer is fine-tuned' protocol is internally inconsistent with the classification heads of ImageNet-pretrained models, and no comparison with alternative stitch-point selection methods is provided. The results therefore do not currently substantiate the title contribution.

major comments (3)
  1. [Section III-B, Eqs. (11)-(12), Algorithm 2, Tables I-II] The fine-tuning protocol is not executable for the Transformer-Transformer and CNN-Transformer experiments. The behind models (DeiT-Ti, DeiT-S, Swin-Ti, Swin-B) are ImageNet-pretrained and their final classification heads output 1000 logits, while the target datasets have 40 (DTD), 37 (Pets) and 10 (ImageNette) classes. If the heads are frozen as stated, the cross-entropy loss in Eq. (12) cannot be computed against the target labels. If the heads are replaced or re-trained, then more than the stitching layer is updated, contradicting the repeated claim 'only the stitching layer is fine-tuned' and the reported trainable-parameter counts (e.g., 0.22M for Swin-B-DeiT-S on DTD). The paper does not describe any head replacement or re-initialization. Consequently, the accuracy and efficiency numbers for these experiments are unsupported.
  2. [Section IV-C, Table III] The default Slow-to-Fast direction is adopted from experiments on a single model pair (ResNet34-VGG11) and a single dataset (CIFAR10). The text states that 'Slow-to-Fast consistently achieves higher accuracy', but Table III contains no repeated trials and no other model/dataset combinations; a general default cannot be established from this evidence.
  3. [Section IV-B, Fig. 4] The central design choice to select stitch points by maximal CKA similarity is validated only by post-hoc plots for a few model pairs. The paper does not report a quantitative correlation (e.g., Spearman rho), confidence intervals, or any baseline such as random stitch-point selection, exhaustive search on a small pair, or a different similarity metric. Without such a comparison, the experiments do not demonstrate that CKA-based selection is better than cheaper alternatives, which is a core part of the claimed contribution.
minor comments (6)
  1. [Algorithm 2, line 2] The frozen tail is written as 'Fixed the parameters of f(1)_{l:j*+1}', but it should refer to f(2)_{l:j*+1}; as written it incorrectly names the front model.
  2. [Section IV-B, Pets paragraph] The text says DeiT-S-DeiT-Ti configurations of 16.80M, 18.13M, and 21.68M parameters reach 92.45%, 92.61%, and 93.62%, but Table I shows the third stitched configuration has 19.02M parameters at 93.92%, while 21.68M is the full DeiT-S at 93.62%.
  3. [Section IV-B, CNN-Transformer paragraph] The text refers to Swin-S as the back model after defining Swin-Ti as the Transformer component; this naming inconsistency should be corrected.
  4. [Table IV] The table is difficult to read: the first row's column headers are garbled and the 'Params' heading is undefined, so the four fine-tuning strategies are not clearly presented.
  5. [Fig. 4] The axes and curve labels are unclear: each subplot has curves called 'Model 1' and 'Model 2' without specifying which is similarity and which is accuracy, so the claimed correlation cannot be quantitatively assessed.
  6. [Section IV-A and Tables I-II] No error bars or multiple runs are reported, and many reported differences are small (e.g., 0.23% on ImageNette); stronger claims such as 'consistently' and 'remarkably' would require repeated trials and variance estimates.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the CKA-based stitching selection and the reported accuracies are measured independently, and the Slow-to-Fast choice is an empirical design decision rather than a consequence of the method's definitions.

full rationale

The paper's central derivation is self-contained. The stitching point is selected by an independently defined criterion, CKA similarity between layer representations (Eq. 7), computed on a batch without consulting downstream task accuracy; the accuracy numbers in Tables I-III are then obtained by actually fine-tuning the stitching layer and measuring classification performance, so they are not equal to the selection criterion by construction. The adoption of the Slow-to-Fast direction is an empirical choice based on the comparison in Table III, not a formal consequence of the CKA computation. Figure 4 reports a post-hoc correlation between CKA similarity and accuracy, but this is an empirical validation rather than a parameter fitted to reproduce the headline results. The self-citations in the related-work section concern pruning and representation-similarity methods and are not load-bearing for the ReStNet algorithm itself. The only notable issue is an experimental-protocol ambiguity for the transformer experiments, where ImageNet-pretrained heads with 1000 output classes appear incompatible with the 37/40/10-class target datasets under the stated 'only the stitching layer is fine-tuned' protocol; however, this is an internal-consistency and correctness concern, not circularity, and therefore does not raise the circularity score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method relies on standard model stitching assumptions plus the CKA heuristic. No new physical or structural entities are introduced. The key free parameters are hyperparameters and the stitching layer design. The axioms are empirical assumptions that are only weakly validated in the paper.

free parameters (5)
  • Learning rate = 0.01 (CNN-CNN), 0.00375 (Transformer and CNN-Transformer)
    Chosen per model family; not derived, stated in Section IV-A.
  • Weight decay = 0.005 (CNN-CNN), 0.05 (Transformer and CNN-Transformer)
    Hyperparameter selected by convention, no justification, Section IV-A.
  • Number of epochs = 150 (CNN-CNN), 300 (Transformer and CNN-Transformer)
    Set without ablation or convergence analysis, Section IV-A.
  • CKA batch size and repeats = batch 256, repeats 5
    Used to compute CKA; stability claim not validated on different batch sizes, Section IV-A.
  • Stitching layer architecture = 1D conv for CNN-CNN, linear for Transformer-Transformer, 2D conv+flatten for CNN-Transformer
    Architectural choice with no comparison of alternatives, Section IV-A.
assumptions (4)
  • domain assumption CKA similarity is a reliable proxy for stitching compatibility and final accuracy.
    Invoked in Section III-B and validated only post-hoc in Section IV-C; no theoretical justification or held-out predictive test.
  • domain assumption Parameter count is the appropriate resource constraint metric for IoT deployment.
    Stated in Section III-B but no memory, latency, or energy measurements are provided to support it.
  • domain assumption A single learnable stitching layer is sufficient to bridge any two pre-trained models and only that layer needs fine-tuning.
    Core to the method, but the paper does not analyze capacity limits, feature mismatches, or the effect of fine-tuning more layers across diverse model pairs.
  • ad hoc to paper Slow-to-Fast (larger model front, smaller model back) is the better stitching direction in general.
    Justified by a single experiment (ResNet34-VGG11 on CIFAR10, Table III); no theoretical reason or wider validation is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReStNet: A Reusable & Stitchable Network for Dynamic Adaptation on IoT Devices." pith.science (2026). https://pith.science/paper/N3CZV42J

@misc{pith2026250609066,
  author       = {Pith},
  title        = {Pith review of: ReStNet: A Reusable & Stitchable Network for Dynamic Adaptation on IoT Devices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N3CZV42J}},
  note         = {Machine review of arXiv:2506.09066}
}
read the original abstract

With the rapid development of deep learning, a growing number of pre-trained models have been publicly available. However, deploying these fixed models in real-world IoT applications is challenging because different devices possess heterogeneous computational and memory resources, making it impossible to deploy a single model across all platforms. Although traditional compression methods, such as pruning, quantization, and knowledge distillation, can improve efficiency, they become inflexible once applied and cannot adapt to changing resource constraints. To address these issues, we propose ReStNet, a Reusable and Stitchable Network that dynamically constructs a hybrid network by stitching two pre-trained models together. Implementing ReStNet requires addressing several key challenges, including how to select the optimal stitching points, determine the stitching order of the two pre-trained models, and choose an effective fine-tuning strategy. To systematically address these challenges and adapt to varying resource constraints, ReStNet determines the stitching point by calculating layer-wise similarity via Centered Kernel Alignment (CKA). It then constructs the hybrid model by retaining early layers from a larger-capacity model and appending deeper layers from a smaller one. To facilitate efficient deployment, only the stitching layer is fine-tuned. This design enables rapid adaptation to changing budgets while fully leveraging available resources. Moreover, ReStNet supports both homogeneous (CNN-CNN, Transformer-Transformer) and heterogeneous (CNN-Transformer) stitching, allowing to combine different model families flexibly. Extensive experiments on multiple benchmarks demonstrate that ReStNet achieve flexible accuracy-efficiency trade-offs at runtime while significantly reducing training cost.

Figures

Figures reproduced from arXiv: 2506.09066 by the authors.

Figure 1
Figure 1. Comparison of traditional fixed models and our ReStNet. (a) shows the traditional deployment method. A single [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustrations of the core challenges of ReStNet. (a) indicates that there are numerous choices for where to stitch between [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Stitching models on different datasets. increases. This again highlights ReStNet’s ability to produce diverse, resource-tailored networks that progressively refine performance. To better illustrate the stitching configuration, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The CKA similarity and corresponding accuracy of the stitched model for different candidate stitching points. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualization of the layer-wise CKA similarity for the base pre-trained models and ReStNet. (a) and (b) show the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 34 canonical work pages

  1. [1]

    Transformers: State- of-the-art natural language processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz et al. , “Transformers: State- of-the-art natural language processing,” in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38–45

    work page 2020

  2. [2]

    Pytorch image models,

    R. Wightman et al. , “Pytorch image models,” 2019

    work page 2019

  3. [3]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  4. [4]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

    arXiv 2014

  5. [5]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 10 012–10 022

    2021

  6. [6]

    Training data-efficient image transformers & distillation through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. J ´egou, “Training data-efficient image transformers & distillation through attention,” in International conference on machine learning . PMLR, 2021, pp. 10 347–10 357

    2021

  7. [7]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    arXiv 2010

  8. [8]

    Davit: Dual attention vision transformers,

    M. Ding, B. Xiao, N. Codella, P. Luo, J. Wang, and L. Yuan, “Davit: Dual attention vision transformers,” in European conference on com- puter vision . Springer, 2022, pp. 74–92

    work page 2022

Show all 56 references
  1. [9]

    Hiera: A hierarchi- cal vision transformer without the bells-and-whistles,

    C. Ryali, Y .-T. Hu, D. Bolya, C. Wei, H. Fan, P.-Y . Huang, V . Aggarwal, A. Chowdhury, O. Poursaeed, J. Hoffman et al. , “Hiera: A hierarchi- cal vision transformer without the bells-and-whistles,” in International conference on machine learning . PMLR, 2023, pp. 29 441–29 454

    2023

  2. [10]

    Scalable vision transformers with hierarchical pooling,

    Z. Pan, B. Zhuang, J. Liu, H. He, and J. Cai, “Scalable vision transformers with hierarchical pooling,” in Proceedings of the IEEE/cvf international conference on computer vision , 2021, pp. 377–386

    2021

  3. [11]

    Understanding the dynamics of dnns using graph modularity,

    Y . Lu, W. Yang, Y . Zhang, Z. Chen, J. Chen, Q. Xuan, Z. Wang, and X. Yang, “Understanding the dynamics of dnns using graph modularity,” in European Conference on Computer Vision . Springer, 2022, pp. 225– 242

    2022

  4. [12]

    A generic layer pruning method for signal modulation recognition deep learning models,

    Y . Lu, Y . Zhu, Y . Li, D. Xu, Y . Lin, Q. Xuan, and X. Yang, “A generic layer pruning method for signal modulation recognition deep learning models,” arXiv preprint arXiv:2406.07929 , 2024

    2024 arXiv

  5. [13]

    Redtest: Towards measuring redundancy in deep neural networks effectively,

    Y . Lu, P. Zhang, J. Wang, L. Ma, X. Yang, and Q. Xuan, “Redtest: Towards measuring redundancy in deep neural networks effectively,” arXiv preprint arXiv:2411.10507 , 2024

    2024 arXiv

  6. [14]

    Sglp: A similarity guided fast layer partition pruning for compressing large deep models,

    Y . Li, Y . Lu, Z. Dong, C. Yang, Y . Chen, and J. Gou, “Sglp: A similarity guided fast layer partition pruning for compressing large deep models,” arXiv preprint arXiv:2410.14720 , 2024

    2024

  7. [15]

    Fcos: A two-stage recoverable model pruning framework for automatic modulation recognition,

    Y . Lu, T. Ma, Z. Wang, Z. Chen, D. Xu, Y . Lin, Q. Xuan, and G. Gui, “Fcos: A two-stage recoverable model pruning framework for automatic modulation recognition,” arXiv preprint arXiv:2505.21571 , 2025

    2025 arXiv

  8. [16]

    Sepprune: Structured pruning for efficient deep speech separation,

    Y . Li, K. Li, X. Yin, Z. Yang, J. Dong, Z. Dong, C. Yang, Y . Tian, and Y . Lu, “Sepprune: Structured pruning for efficient deep speech separation,” arXiv preprint arXiv:2505.12079 , 2025

    2025 arXiv

  9. [17]

    Reassessing layer pruning in llms: New insights and methods,

    Y . Lu, H. Cheng, Y . Fang, Z. Wang, J. Wei, D. Xu, Q. Xuan, X. Yang, and Z. Zhu, “Reassessing layer pruning in llms: New insights and methods,” arXiv preprint arXiv:2411.15558 , 2024

    2024 arXiv

  10. [19]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015

    2015 arXiv

  11. [20]

    A semi-supervised federated learning scheme via knowledge distillation for intrusion detection,

    R. Zhao, L. Yang, Y . Wang, Z. Xue, G. Gui, and T. Ohtsuki, “A semi-supervised federated learning scheme via knowledge distillation for intrusion detection,” in ICC 2022-IEEE International Conference on Communications. IEEE, 2022, pp. 2688–2693

    2022

  12. [21]

    Knowledge distillation: A survey,

    J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision , vol. 129, no. 6, pp. 1789–1819, 2021

    2021

  13. [22]

    On the efficacy of knowledge distillation,

    J. H. Cho and B. Hariharan, “On the efficacy of knowledge distillation,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 4794–4802

    2019

  14. [23]

    Knowledge distillation from a stronger teacher,

    T. Huang, S. You, F. Wang, C. Qian, and C. Xu, “Knowledge distillation from a stronger teacher,” Advances in Neural Information Processing Systems, vol. 35, pp. 33 716–33 727, 2022

    2022

  15. [24]

    Q-vit: Accurate and fully quantized low-bit vision transformer,

    Y . Li, S. Xu, B. Zhang, X. Cao, P. Gao, and G. Guo, “Q-vit: Accurate and fully quantized low-bit vision transformer,” Advances in neural information processing systems , vol. 35, pp. 34 451–34 463, 2022

    2022

  16. [25]

    Ptqd: Accurate post-training quantization for diffusion models,

    Y . He, L. Liu, J. Liu, W. Wu, H. Zhou, and B. Zhuang, “Ptqd: Accurate post-training quantization for diffusion models,” arXiv preprint arXiv:2305.10657, 2023

    2023 arXiv

  17. [26]

    Bit- shrinking: Limiting instantaneous sharpness for improving post-training quantization,

    C. Lin, B. Peng, Z. Li, W. Tan, Y . Ren, J. Xiao, and S. Pu, “Bit- shrinking: Limiting instantaneous sharpness for improving post-training quantization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 16 196–16 205

    2023

  18. [27]

    Post-training quantization for vision transformer,

    Z. Liu, Y . Wang, K. Han, W. Zhang, S. Ma, and W. Gao, “Post-training quantization for vision transformer,” Advances in Neural Information Processing Systems, vol. 34, pp. 28 092–28 103, 2021

    2021

  19. [28]

    Zeroq: A novel zero shot quantization framework,

    Y . Cai, Z. Yao, Z. Dong, A. Gholami, M. W. Mahoney, and K. Keutzer, “Zeroq: A novel zero shot quantization framework,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 13 169–13 178

    2020

  20. [29]

    Quantization networks,

    J. Yang, X. Shen, J. Xing, X. Tian, H. Li, B. Deng, J. Huang, and X.-s. Hua, “Quantization networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 7308– 7316

    2019

  21. [30]

    Understanding image representations by measuring their equivariance and equivalence,

    K. Lenc and A. Vedaldi, “Understanding image representations by measuring their equivariance and equivalence,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2015

    2015

  22. [31]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” Handbook of Systemic Autoimmune Diseases , vol. 1, no. 4, 2009

    2009

  23. [32]

    Describing textures in the wild,

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi, “Describing textures in the wild,” in2014 IEEE Conference on Computer Vision and Pattern Recognition , 2014, pp. 3606–3613

    2014

  24. [33]

    Cats and dogs,

    O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. V . Jawahar, “Cats and dogs,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 3498–3505

    2012

  25. [34]

    Imagenette: A smaller subset of 10 easily classified classes from imagenet,

    J. Howard, “Imagenette: A smaller subset of 10 easily classified classes from imagenet,” March 2019. [Online]. Available: https: //github.com/fastai/imagenette

    2019

  26. [35]

    Performance-optimized hierarchical models predict neural responses in higher visual cortex,

    D. L. K. Yamins, H. Hong, C. F. Cadieu, E. A. Solomon, D. Seibert, and J. J. Dicarlo, “Performance-optimized hierarchical models predict neural responses in higher visual cortex,” Proceedings of the National Academy of Sciences , vol. 111, no. 23, pp. 8619–24, 2014

    2014

  27. [36]

    Canonical correlation analysis: An overview with application to learning methods,

    D. R. Hardoon, S. Szedmak, and J. Shawe-Taylor, “Canonical correlation analysis: An overview with application to learning methods,” Neural Computation, vol. 16, no. 12, pp. 2639–2664, 2004

    2004

  28. [37]

    Svcca: singular vector canonical correlation analysis for deep learning dynamics and interpretability,

    M. Raghu, J. Gilmer, J. Yosinski, and J. Sohl-Dickstein, “Svcca: singular vector canonical correlation analysis for deep learning dynamics and interpretability,” in Proceedings of the 31st International Conference on Neural Information Processing Systems , ser. NIPS’17, 2017, ...

    2017

  29. [38]

    Similarity of neural network representations revisited,

    S. Kornblith, M. Norouzi, H. Lee, and G. Hinton, “Similarity of neural network representations revisited,” in Proceedings of the 36th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97....

    2019

  30. [39]

    Graph- based similarity of deep neural networks,

    Z. Chen, Y . Lu, J. Hu, Q. Xuan, Z. Wang, and X. Yang, “Graph- based similarity of deep neural networks,” Neurocomputing, vol. 614, p. 128722, 2025

    2025

  31. [40]

    Similarity-preserving knowledge distillation,

    F. Tung and G. Mori, “Similarity-preserving knowledge distillation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019

    2019

  32. [41]

    Deconfounded repre- sentation similarity for comparison of neural networks,

    T. Cui, Y . Kumar, P. Marttinen, and S. Kaski, “Deconfounded repre- sentation similarity for comparison of neural networks,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, ser. NIPS ’22. Red Hook, NY , USA: Curran Associates Inc., 2022

    2022

  33. [42]

    Representation similarity analysis for efficient task taxonomy & transfer learning,

    K. Dwivedi and G. Roig, “Representation similarity analysis for efficient task taxonomy & transfer learning,” in Proceedings of the IEEE/CVF JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 10 Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019

    2021

  34. [43]

    Similarity and matching of neural network representations,

    A. Csisz ´arik, P. K ˝or¨osi-Szab´o, A. K. Matszangosz, G. Papp, and D. Varga, “Similarity and matching of neural network representations,” in Proceedings of the 35th International Conference on Neural Informa- tion Processing Systems , ser. NIPS ’21. Red Hook, NY , USA: Curra...

    2021

  35. [44]

    Revisiting model stitching to compare neural representations,

    Y . Bansal, P. Nakkiran, and B. Barak, “Revisiting model stitching to compare neural representations,” in Proceedings of the 35th Interna- tional Conference on Neural Information Processing Systems , ser. NIPS ’21. Red Hook, NY , USA: Curran Associates Inc., 2021

    2021

  36. [45]

    How to train your vit? data, augmentation, and regularization in vision transformers,

    A. Steiner, A. Kolesnikov, , X. Zhai, R. Wightman, J. Uszkoreit, and L. Beyer, “How to train your vit? data, augmentation, and regularization in vision transformers,” arXiv preprint arXiv:2106.10270 , 2021

    2021 arXiv

  37. [46]

    Stitchable neural networks,

    Z. Pan, J. Cai, and B. Zhuang, “Stitchable neural networks,” in CVPR, 2023

    2023

  38. [47]

    Measuring statistical dependence with hilbert-schmidt norms,

    A. Gretton, O. Bousquet, A. Smola, and B. Sch ¨olkopf, “Measuring statistical dependence with hilbert-schmidt norms,” in International conference on algorithmic learning theory . Springer, 2005, pp. 63– 77

    2005

  39. [48]

    Low-resource scenario classification through model pruning towards refined edge intelligence,

    X. Shan, J. Wang, X. Yan, C. Wang, X. Zhang, G. Gui, and H. Sari, “Low-resource scenario classification through model pruning towards refined edge intelligence,” IEEE Internet of Things Journal , 2023

    2023

  40. [49]

    Mobilenet and knowledge distillation-based automatic scenario recognition method in vehicle-to- vehicle systems,

    J. Yang, Y . Wang, H. Zhao, and G. Gui, “Mobilenet and knowledge distillation-based automatic scenario recognition method in vehicle-to- vehicle systems,” IEEE Transactions on V ehicular Technology, vol. 71, no. 10, pp. 11 006–11 016, 2022

    2022

  41. [50]

    An improved neural network pruning technology for automatic modulation classification in edge devices,

    Y . Lin, Y . Tu, and Z. Dou, “An improved neural network pruning technology for automatic modulation classification in edge devices,” IEEE Transactions on V ehicular Technology , vol. 69, no. 5, pp. 5703– 5706, 2020

    2020

  42. [51]

    Glr-sei: green and low resource specific emitter identification based on complex networks and fisher pruning,

    Y . Lin, H. Zha, Y . Tu, S. Zhang, W. Yan, and C. Xu, “Glr-sei: green and low resource specific emitter identification based on complex networks and fisher pruning,” IEEE transactions on emerging topics in computational intelligence , 2023

    2023

  43. [52]

    Rgp: Neural network pruning through regular graph with edges swap- ping,

    Z. Chen, J. Xiang, Y . Lu, Q. Xuan, Z. Wang, G. Chen, and X. Yang, “Rgp: Neural network pruning through regular graph with edges swap- ping,” IEEE Transactions on Neural Networks and Learning Systems , 2023

    2023

  44. [53]

    Lightweight automatic modulation classification via progres- sive differentiable architecture search,

    X. Zhang, X. Chen, Y . Wang, G. Gui, B. Adebisi, H. Sari, and F. Adachi, “Lightweight automatic modulation classification via progres- sive differentiable architecture search,” IEEE Transactions on Cognitive Communications and Networking , 2023

    2023

  45. [54]

    Complex-valued networks for automatic modulation classification,

    Y . Tu, Y . Lin, C. Hou, and S. Mao, “Complex-valued networks for automatic modulation classification,” IEEE Transactions on V ehicular Technology, vol. 69, no. 9, pp. 10 085–10 089, 2020

    2020

  46. [55]

    Surgical fine-tuning improves adaptation to distribution shifts,

    Y . Lee, A. S. Chen, F. Tajwar, A. Kumar, H. Yao, P. Liang, and C. Finn, “Surgical fine-tuning improves adaptation to distribution shifts,” arXiv preprint arXiv:2210.11466, 2022

    2022 arXiv

  47. [56]

    Backdoor pre-trained models can transfer to all,

    L. Shen, S. Ji, X. Zhang, J. Li, J. Chen, J. Shi, C. Fang, J. Yin, and T. Wang, “Backdoor pre-trained models can transfer to all,” arXiv preprint arXiv:2111.00197, 2021

    2021 arXiv

  48. [57]

    Semantic rela- tion reasoning for shot-stable few-shot object detection,

    C. Zhu, F. Chen, U. Ahmed, Z. Shen, and M. Savvides, “Semantic rela- tion reasoning for shot-stable few-shot object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 8782–8791. Maoyu Wang is currently pursuing the B.S....

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.