Pith. sign in

REVIEW 5 major objections 5 minor 133 references

ResidualDroppath: Enhancing Feature Reuse over Residual Connections

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that alternating droppath iterations with iterations that train only the dropped residual paths makes residual networks reuse features as identity mappings and improves image-classification accuracy.

desk verdict New training trick, one solid CIFAR-10 number, and a motivation that doesn't survive close reading—worth a look but needs work. read the letter →

arxiv 2411.09475 v1 pith:24CQXMSY submitted 2024-11-14 cs.LG cs.AI

Sejik Park This is my paper · ORCID
classification cs.LGcs.AI MSC 68T07
keywords featurereuseresidualconnectionsdroppathstochasticdepthimageclassificationidentitymappingtrainingalgorithminformationretention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Residual connections are widely credited with letting deep networks reuse features, but this paper argues that vanilla residual training does not actually encourage the network to pass features through unchanged. Using a toy spiral dataset and a visualization tool, the author shows that MLPs with residual connections tend to transform features repeatedly, producing similar distributions at non-adjacent layers and losing information. To counter this, the paper proposes ResidualDroppath, a training scheme that alternates between droppath iterations (which randomly drop some residual paths, forcing the model to rely on identity shortcuts) and iterations that train only the newly dropped paths while freezing the undropped ones. On CIFAR-10, MNIST, and ImageNet-1k, applying this scheme to ResNet50 and ResNet50d improves Top-1/Top-5 accuracy in most settings, with the largest gain on CIFAR-10. The paper's own limitation is that ResNet50 on ImageNet-1k does not beat the plain droppath baseline.

What carries the argument

The central object is the ResidualDroppath training loop, which alternates two iteration types per block. Even iterations apply a droppath mask $m_d$ so that the output is $x + F(x) \cdot m_d$, forcing dropped positions to pass the input $x$ unchanged. Odd iterations compute $x + F(x).\mathrm{detach}() \cdot m_d + F(x) \cdot (1 - m_d)$, which detaches (freezes) the kept paths and trains only the dropped ones. This two-step alternation is designed to teach the residual branch $F$ to produce updates that leave features reusable in their identity form.

What would settle it

Train ResNet50d on CIFAR-10 with and without ResidualDroppath under matched compute (same total gradient updates) and measure the average cosine similarity between each block's input and output on the test set. If the ResidualDroppath model does not show a higher fraction of blocks whose output is close to their input compared with the droppath baseline, the claimed identity-reuse mechanism is not what drives the accuracy improvement.

Watch

Extended reading notes

Core claim

Residual connections alone do not guarantee efficient feature reuse: even with a skip connection, intermediate layers keep applying transformations to features that could be passed through in identity form, and similar feature distributions reappear in non-consecutive layers, which risks information loss. The paper's proposed ResidualDroppath addresses this by alternating two training iterations. In the first, droppath randomly drops a subset of the residual path, so the block output reduces to the identity (the input) for dropped positions and the network must learn under forced feature reuse. In the second, the paths that were not dropped are frozen and the dropped parts are trained, so those residual branches learn outputs that are compatible with identity reuse. The result is a training-time modification that improves image-classification accuracy for ResNet50/ResNet50d on CIFAR-10 and MNIST, and for ResNet50d on ImageNet-1k, with no change at inference.

Load-bearing premise

The load-bearing premise is that the feature-reuse deficiency observed in small MLPs trained on a 2D spiral toy dataset generalizes to deep convolutional residual networks trained on natural images.

Editorial extensions

If this is right

  • If ResidualDroppath works as claimed, any residual network can be trained with a simple two-stage alternation at no extra inference cost, and the accuracy gains on CIFAR-10 (~1.3 points over the standard ResNet50d baseline) indicate the mechanism is most effective when per-class information is abundant.
  • The method suggests that feature reuse is not an automatic property of skip connections but a learnable behavior, so training that explicitly forces identity reuse should be part of a residual network's optimization.
  • Because the algorithm is architecture-agnostic in spirit, it may extend to other residual architectures (e.g., transformers) as the paper notes as future work.
  • On ImageNet-1k, the gains are smaller and the ResNet50 case loses to droppath, so the benefit appears to depend on the information richness of the dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The toy-MLP analysis uses Euclidean distance between input and layer features as a proxy for feature reuse; a direct test on natural-image ResNet activations (e.g., CKA or cosine similarity between block input and output) would let us check whether the proposed mechanism really increases identity reuse in the trained ResNets.
  • The second iteration doubles the number of forward/backward passes per block; if the gains come mainly from the extra optimization steps or from the detach operation alone, a simpler scheme might produce the same result. Comparing ResidualDroppath against an equal-compute droppath baseline would isolate the alternation's contribution.
  • The interpretation of dataset information content—why CIFAR-10 gains are large, MNIST small, and ImageNet mixed—could be tested by ablating the number of classes or images per class while keeping the architecture fixed.
  • The paper's insight that residual training may over-transform features connects to a broader design space of training-time regularizers that encourage shortcut use, beyond the specific droppath mask.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper argues that vanilla residual connections limit feature reuse across non-consecutive layers, supports this claim with visualizations of a toy MLP trained on a 2D spiral dataset, and proposes ResidualDroppath, a training procedure that alternates between droppath-style iterations and iterations that freeze the non-dropped residual paths. The method is evaluated on MNIST, CIFAR-10, and ImageNet-1k with ResNet50 and ResNet50d. The authors report the strongest gains on CIFAR-10 (around +1.3% top-1 accuracy over both standard training and droppath), small gains on MNIST, and mixed or marginal results on ImageNet-1k, where the ResNet50 result is worse than droppath and ResNet50d improves only slightly.

Significance. If the CIFAR-10 result is reproducible, ResidualDroppath offers a simple, training-time accuracy improvement for residual networks with no change at inference. The algorithm is clearly specified and the experimental setup is mostly standard. However, the paper's central mechanistic claim—that the method improves feature reuse—is not directly tested in the ResNet experiments, and the toy-analysis evidence for the initial deficiency is weakened by the use of a scale-sensitive similarity measure. The ImageNet results lack error bars and are inconsistent across architectures. The paper is honest about its limitations, but the gap between the claimed mechanism and the empirical evidence is the main weakness.

major comments (5)
  1. [Section 3.1, Figure 3] The Euclidean-distance similarity measure used in Figure 3 is not scale-normalized. Activation magnitudes can vary substantially across layers in a 32-layer residual MLP, so a rise-and-fall in Euclidean distance may track changes in feature norm rather than a genuine absence of feature reuse. The claim that vanilla residual connections produce a 'feature reuse deficiency' is therefore not established by the presented evidence. Please recompute the analysis with a scale-invariant measure (e.g., cosine similarity or centered kernel alignment) and, ideally, compare against a non-residual baseline or a null model.
  2. [Section 4.2, Table 1] The ImageNet-1k results are from a single seed with no error bars. For ResNet50, ResidualDroppath (75.79 top-1) is worse than Droppath (76.11), and for ResNet50d the gain over Droppath is only 0.08 top-1. The abstract and introduction claim 'significant improvement' on MNIST and CIFAR-10; the MNIST differences (0.06% with a standard deviation of 0.05%) are within noise. The only statistically solid result is CIFAR-10. Please either restrict the claims to CIFAR-10 or provide multi-seed ImageNet results with error bars that support the stronger conclusion.
  3. [Algorithm 1, line 16] The second iteration (M % 2 == 1) reuses the mask variable 'maskd' sampled in the previous iteration, but this is not stated in the algorithm or the text. More importantly, the paper provides no measurement of feature reuse in the trained ResNet models, so the mechanism claimed in the title and Section 1—that ResidualDroppath 'facilitates feature reuse across multiple layers'—remains untested. Please add a direct feature-reuse analysis (e.g., layer-wise similarity or CKA on the actual trained models with and without ResidualDroppath) to support the mechanistic interpretation.
  4. [Section 3.1] The toy analysis trains MLPs with GELU activations on a 2D spiral dataset and interprets layer-wise feature similarity as evidence for the behavior of deep convolutional residual networks on natural images. This transfer is assumed but not demonstrated. Without evidence that the same pattern appears in ResNet blocks, the motivation for the algorithm is weakened. Please either provide a similar analysis for a small ResNet on a natural-image subset or explicitly reframe the toy analysis as illustrative rather than diagnostic.
  5. [Section 4.1] The drop rate (0.1) and the alternation period (every two iterations) are fixed hyperparameters, and no sensitivity analysis is reported. Since these are the only free parameters introduced by the method, their robustness is load-bearing for the reproducibility of the claimed gains. Please include an ablation over at least a small range of drop rates and alternation periods.
minor comments (5)
  1. [Section 4.1] The baseline description contains the typo 'ResidaulDroppath'; should read 'ResidualDroppath'.
  2. [Section 3.1] The text says 'In this session' but should say 'In this section'.
  3. [Table 1] The dataset column header uses 'Imagenet-1k' while the body text uses 'ImageNet-1k'; please be consistent.
  4. [Figure 3] The caption states that 'similarity decreases and increases' but does not state whether higher heatmap values indicate higher or lower similarity; clarify the color scale.
  5. [References] Reference [1] is marked 'under review' and is not a stable archival reference; please replace it with a citable version or remove it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: experimental claims are benchmarked externally; the toy analysis is motivation, not a fitting device, and self-citations are not load-bearing.

full rationale

The paper's core claim is empirical: ResidualDroppath improves test accuracy relative to standard training and Droppath on MNIST, CIFAR-10, and ImageNet-1K (Table 1). Those results are compared against external baselines, and no hyperparameter or parameter of the method is fitted to the target accuracies; the drop rate is fixed at 0.1, matching the Droppath default setting from PyTorch Image Models. The Section 3.1 toy visualization is motivational: it suggests that residual connections may cause repeated transformations and thus inefficient reuse, but the algorithm's design (alternating two iteration types) is a stated design choice, not a value inferred from the experimental outcomes. The self-citations [82,83] appear only in supplementary related-work discussion of feature exploration and are not load-bearing for the main derivation. The paper also discloses limitations (ResNet50 does not beat Droppath on ImageNet-1K; intra-layer redundant nodes are ignored), which are consistent with an honest empirical report rather than a circular argument. The Euclidean-distance similarity metric concerns (scale sensitivity) and the absence of a direct feature-reuse measurement in the ResNet experiments are evidentiary weaknesses, not circularity, because the accuracy gains are independent of the toy diagnosis. No step in the paper reduces by construction to its input; no fitted parameter is relabeled as a prediction; no load-bearing premise rests on a self-citation chain.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the interpretation of the toy feature analysis and the assumption that the two-phase training improves feature reuse. No fitted constants or invented entities are introduced, but the drop rate and alternation period are hand-chosen, and the transferability of the analysis is unverified.

free parameters (2)
  • Drop rate for ResidualDroppath mask = not stated in paper (Droppath baseline uses 0.1)
    The mask probability controls how many residual paths are dropped in the first iteration type; the paper does not report the value used for the proposed method, only that scaling is not applied.
  • Alternation period = every other iteration (1:1 ratio)
    The training alternates between droppath and frozen-path updates every step; the ratio is chosen by hand and not varied.
assumptions (3)
  • domain assumption Residual connections aid training by promoting feature reuse
    The motivation in Section 1 interprets residual connections through feature reuse and assumes maximizing reuse is beneficial.
  • domain assumption Similar feature distributions in non-consecutive layers indicate inefficient feature reuse and information loss
    Section 3.1 interprets the Euclidean distance and contour plots as evidence of transformation burden; this interpretation is not quantitatively established.
  • ad hoc to paper Toy MLP analysis on a spiral dataset transfers to deep ResNet models on real images
    The algorithm is designed from observations on 6- and 32-layer MLPs trained on a 2D spiral dataset, then applied to ResNet50/50d; no evidence is given that the same feature-reuse deficiency occurs in convolutional residual networks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ResidualDroppath: Enhancing Feature Reuse over Residual Connections." pith.science (2026). https://pith.science/paper/24CQXMSY

@misc{pith2026241109475,
  author       = {Pith},
  title        = {Pith review of: ResidualDroppath: Enhancing Feature Reuse over Residual Connections},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/24CQXMSY}},
  note         = {Machine review of arXiv:2411.09475}
}
read the original abstract

Residual connections are one of the most important components in neural network architectures for mitigating the vanishing gradient problem and facilitating the training of much deeper networks. One possible explanation for how residual connections aid deeper network training is by promoting feature reuse. However, we identify and analyze the limitations of feature reuse with vanilla residual connections. To address these limitations, we propose modifications in training methods. Specifically, we provide an additional opportunity for the model to learn feature reuse with residual connections through two types of iterations during training. The first type of iteration involves using droppath, which enforces feature reuse by randomly dropping a subset of layers. The second type of iteration focuses on training the dropped parts of the model while freezing the undropped parts. As a result, the dropped parts learn in a way that encourages feature reuse, as the model relies on the undropped parts with feature reuse in mind. Overall, we demonstrated performance improvements in models with residual connections for image classification in certain cases.

Figures

Figures reproduced from arXiv: 2411.09475 by the authors.

Figure 1
Figure 1. Feature Reuse Across Multiple Layers. It visualizes the feature distribution of a model trained with linear layers of depth 32 and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Toy Dataset. It visualizes 400 sampled points from the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Feature Visualization during Training with a Model [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: ResidualDroppath. It visualizes the operation of our proposed algorithm at the block level. There are two iteration stages. In the [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

133 extracted references · 39 canonical work pages

  1. [1]

    Layer by layer: Uncovering where multi-task learning happens in instruction-tuned large language mod- els

    Anonymous. Layer by layer: Uncovering where multi-task learning happens in instruction-tuned large language mod- els. In Submitted to ACL Rolling Review - June 2024, 2024. under review. 1

    2024

  2. [2]

    Understanding and improving features learned in deep functional maps

    Souhaib Attaiki and Maks Ovsjanikov. Understanding and improving features learned in deep functional maps. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1316–1326, 2023. 1

    2023

  3. [3]

    Information complexity of stochastic convex optimization: Applications to generaliza- tion, memorization, and tracing

    Idan Attias, Gintare Karolina Dziugaite, Mahdi Haghifam, Roi Livni, and Daniel M Roy. Information complexity of stochastic convex optimization: Applications to generaliza- tion, memorization, and tracing. In Forty-first International Conference on Machine Learning. 1

  4. [4]

    Does roberta perform better than bert in continual learning: An attention sink perspective

    Xueying Bai, Yifan Sun, and Niranjan Balasubramanian. Does roberta perform better than bert in continual learning: An attention sink perspective. In First Conference on Lan- guage Modeling. 1

  5. [5]

    Layer swapping for zero-shot cross-lingual transfer in large lan- guage models

    Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, Nayan Singhal, Hongjiang Lv, and Bing Liu. Layer swapping for zero-shot cross-lingual transfer in large lan- guage models. arXiv preprint arXiv:2410.01335, 2024. 1

    arXiv 2024

  6. [6]

    Avoiding mode collapse in diffusion models fine-tuned with reinforcement learning

    Roberto Barcel ´o, Crist ´obal Alc ´azar, and Felipe Tobar. Avoiding mode collapse in diffusion models fine-tuned with reinforcement learning. arXiv preprint arXiv:2410.08315,

    arXiv

  7. [7]

    Network dissection: Quantifying inter- pretability of deep visual representations

    David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying inter- pretability of deep visual representations. InProceedings of the IEEE conference on computer vision and pattern recog- nition, pages 6541–6549, 2017. 1

    2017

  8. [8]

    Gan dissection: Visualizing and under- standing generative adversarial networks

    David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B Tenenbaum, William T Freeman, and Anto- nio Torralba. Gan dissection: Visualizing and under- standing generative adversarial networks. arXiv preprint arXiv:1811.10597, 2018. 1

    arXiv 2018

Show all 133 references
  1. [9]

    The heuristic core: Understanding subnetwork generaliza- tion in pretrained language models

    Adithya Bhaskar, Dan Friedman, and Danqi Chen. The heuristic core: Understanding subnetwork generaliza- tion in pretrained language models. arXiv preprint arXiv:2403.03942, 2024. 1

    2024 arXiv

  2. [10]

    Mixing it up: The cocktail effect of multi-task fine-tuning on llm performance–a case study in finance

    Meni Brief, Oded Ovadia, Gil Shenderovitz, Noga Ben Yoash, Rachel Lemberg, and Eitam Sheetrit. Mixing it up: The cocktail effect of multi-task fine-tuning on llm performance–a case study in finance. arXiv preprint arXiv:2410.01109, 2024. 1

    arXiv

  3. [11]

    Exploration by random network distillation

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. arXiv preprint arXiv:1810.12894, 2018. 1

    2018 arXiv

  4. [12]

    Adashift: Learning discriminative self-gated neural feature activation with an adaptive shift factor

    Sudong Cai. Adashift: Learning discriminative self-gated neural feature activation with an adaptive shift factor. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 5947–5956, 2024. 1

    2024

  5. [13]

    Sorsa: Singular values and orthonormal regular- ized singular vectors adaptation of large language models

    Yang Cao. Sorsa: Singular values and orthonormal regular- ized singular vectors adaptation of large language models. arXiv preprint arXiv:2409.00055, 2024. 1

    2024 arXiv

  6. [14]

    Dsg-kd: Knowledge distillation from domain-specific to general language models

    Sangyeon Cho, Jangyeong Jeon, Dongjoon Lee, Changhee Lee, and Junyeong Kim. Dsg-kd: Knowledge distillation from domain-specific to general language models. IEEE Access, 2024. 1

    2024

  7. [15]

    Randaugment: Practical automated data augmen- tation with a reduced search space

    Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmen- tation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702–703, 2020. 1

    2020

  8. [16]

    Vision transformers need registers

    Timoth ´ee Darcet, Maxime Oquab, Julien Mairal, and Pi- otr Bojanowski. Vision transformers need registers. arXiv preprint arXiv:2309.16588, 2023. 1

    2023 arXiv

  9. [17]

    Sparse autoencoders reveal tem- poral difference learning in large language models

    Can Demircan, Tankred Saanum, Akshay K Jagadish, Mar- cel Binz, and Eric Schulz. Sparse autoencoders reveal tem- poral difference learning in large language models. arXiv preprint arXiv:2410.01280, 2024. 2

    2024 arXiv

  10. [18]

    Why fine-tuning strug- gles with forgetting in machine unlearning? theoretical in- sights and a remedial approach, 2024

    Meng Ding, Jinhui Xu, and Kaiyi Ji. Why fine-tuning strug- gles with forgetting in machine unlearning? theoretical in- sights and a remedial approach, 2024. 1

    2024

  11. [19]

    Decaf: A deep convolutional activation feature for generic visual recog- nition

    Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recog- nition. In International conference on machine learning , pages 647–655. PMLR, 2014. 1

    2014

  12. [20]

    Incorporating nesterov momentum into adam

    Timothy Dozat. Incorporating nesterov momentum into adam. 2016. 4

    2016

  13. [21]

    Rosetta neurons: Mining the common units in a model zoo

    Amil Dravid, Yossi Gandelsman, Alexei A Efros, and Assaf Shocher. Rosetta neurons: Mining the common units in a model zoo. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1934–1943, 2023. 2, 1

    1934

  14. [22]

    Compositional genera- tive modeling: A single model is not all you need

    Yilun Du and Leslie Kaelbling. Compositional genera- tive modeling: A single model is not all you need. arXiv preprint arXiv:2402.01103, 2024. 1

    2024 arXiv

  15. [23]

    Content-adaptive non-local convolution for remote sensing pansharpening

    Yule Duan, Xiao Wu, Haoyu Deng, and Liang-Jian Deng. Content-adaptive non-local convolution for remote sensing pansharpening. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 27738–27747, 2024. 1

    2024

  16. [24]

    Roma: Robust dense feature matching

    Johan Edstedt, Qiyu Sun, Georg B ¨okman, M ˚arten Wadenb¨ack, and Michael Felsberg. Roma: Robust dense feature matching. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 19790–19800, 2024. 1

    2024

  17. [25]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machin...

  18. [26]

    Tasl: Task skill localiza- tion and consolidation for language model continual learn- ing

    Yujie Feng, Xu Chu, Yongxin Xu, Zexin Lu, Bo Liu, Philip S Yu, and Xiao-Ming Wu. Tasl: Task skill localiza- tion and consolidation for language model continual learn- ing. arXiv preprint arXiv:2408.05200, 2024

    2024 arXiv

  19. [27]

    Sharpness-aware minimization for efficiently improving generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. In International Conference on Learning Representations, 2021. 1

    2021

  20. [28]

    Enhancing elu- sive clues in knowledge learning by contrasting attention of language models

    Jian Gao, Xiao Zhang, Ji Wu, and Miao Li. Enhancing elu- sive clues in knowledge learning by contrasting attention of language models. arXiv preprint arXiv:2409.17954, 2024. 1

    2024 arXiv

  21. [29]

    Build- ing a subspace of policies for scalable continual learning

    Jean-Baptiste Gaya, Thang Doan, Lucas Caccia, Laure Soulier, Ludovic Denoyer, and Roberta Raileanu. Build- ing a subspace of policies for scalable continual learning. arXiv preprint arXiv:2211.10445, 2022. 1

    2022 arXiv

  22. [30]

    What do vision transformers learn? a visual exploration

    Amin Ghiasi, Hamid Kazemi, Eitan Borgnia, Steven Reich, Manli Shu, Micah Goldblum, Andrew Gordon Wilson, and Tom Goldstein. What do vision transformers learn? a visual exploration. arXiv preprint arXiv:2212.06727, 2022. 1

    2022 arXiv

  23. [31]

    Dropblock: A regularization method for convolutional networks

    Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. Dropblock: A regularization method for convolutional networks. Ad- vances in neural information processing systems, 31, 2018. 2

    2018

  24. [32]

    Task-adaptive pretrained language models via clustered- importance sampling, 2024

    David Grangier, Simin Fan, Skyler Seto, and Pierre Ablin. Task-adaptive pretrained language models via clustered- importance sampling, 2024. 1

    2024

  25. [33]

    Uncovering unique concept vectors through latent space decomposition

    Mara Graziani, Laura O’ Mahony, An-Phi Nguyen, Hen- ning M¨uller, and Vincent Andrearczyk. Uncovering unique concept vectors through latent space decomposition. arXiv preprint arXiv:2307.06913, 2023. 2, 1

    2023 arXiv

  26. [34]

    Preserving linear separability in continual learning by backward fea- ture projection

    Qiao Gu, Dongsub Shim, and Florian Shkurti. Preserving linear separability in continual learning by backward fea- ture projection. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 24286–24295, 2023. 1

    2023

  27. [35]

    Cpp-net: Embracing multi- scale feature fusion into deep unfolding cp-ppa network for compressive sensing

    Zhen Guo and Hongping Gan. Cpp-net: Embracing multi- scale feature fusion into deep unfolding cp-ppa network for compressive sensing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 25086–25095, 2024. 1

    2024

  28. [36]

    Slim: Let llm learn more and forget less with soft lora and identity mixture, 2024

    Jiayi Han, Liang Du, Hongwei Du, Xiangguo Zhou, Yiwen Wu, Weibo Zheng, and Donghong Han. Slim: Let llm learn more and forget less with soft lora and identity mixture, 2024

    2024

  29. [37]

    Upcycling large language models into mixture of experts, 2024

    Ethan He, Abhinav Khattar, Ryan Prenger, Vijay Kor- thikanti, Zijie Yan, Tong Liu, Shiqing Fan, Ashwath Aithal, Mohammad Shoeybi, and Bryan Catanzaro. Upcycling large language models into mixture of experts, 2024. 1

    2024

  30. [38]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 1, 5

    2016

  31. [39]

    Identity mappings in deep residual networks

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Computer Vision–ECCV 2016: 14th European Conference, Amster- dam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 630–645. Springer, 2016. 1

    2016

  32. [40]

    Debi- asing text-to-image diffusion models

    Ruifei He, Chuhui Xue, Haoru Tan, Wenqing Zhang, Yingchen Yu, Song Bai, and Xiaojuan Qi. Debi- asing text-to-image diffusion models. arXiv preprint arXiv:2402.14577, 2024. 1

    2024 arXiv

  33. [41]

    Bag of tricks for image classifica- tion with convolutional neural networks

    Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Jun- yuan Xie, and Mu Li. Bag of tricks for image classifica- tion with convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 558–567, 2019. 5

    2019

  34. [42]

    Gaussian error linear units (gelus)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016. 4

    2016 arXiv

  35. [43]

    Augmix: A simple data processing method to improve robustness and uncertainty

    Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781, 2019. 1

    1912 arXiv

  36. [44]

    From tinkering to engineering: Measurements in tensorflow playground

    Henrik Hoeiness, Axel Harstad, and Gerald Friedland. From tinkering to engineering: Measurements in tensorflow playground. arXiv preprint arXiv:2101.04141, 2021. 1, 2, 3

    2021 arXiv

  37. [45]

    Concept-centric transformers: Enhancing model inter- pretability through object-centric concept learning within a shared global workspace

    Jinyung Hong, Keun Hee Park, and Theodore P Pavlic. Concept-centric transformers: Enhancing model inter- pretability through object-centric concept learning within a shared global workspace. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , ...

    2024

  38. [46]

    Deep networks with stochastic depth

    Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kil- ian Q Weinberger. Deep networks with stochastic depth. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Pro- ceedings, Part IV 14, pages 646–661. Springer, 2016. 1, 2, 5

    2016

  39. [47]

    Densely connected convolutional net- works

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kil- ian Q Weinberger. Densely connected convolutional net- works. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017. 1, 2

    2017

  40. [48]

    The platonic representation hypothesis

    Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis. arXiv preprint arXiv:2405.07987, 2024. 1

    2024 arXiv

  41. [49]

    Comparing the decision-making mechanisms by transformers and cnns via explanation methods

    Mingqi Jiang, Saeed Khorram, and Li Fuxin. Comparing the decision-making mechanisms by transformers and cnns via explanation methods. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9546–9555, 2024. 1

    2024

  42. [50]

    No train no gain: Revisiting ef- ficient training algorithms for transformer-based language models

    Jean Kaddour, Oscar Key, Piotr Nawrot, Pasquale Min- ervini, and Matt J Kusner. No train no gain: Revisiting ef- ficient training algorithms for transformer-based language models. Advances in Neural Information Processing Sys- tems, 36, 2024. 1

    2024

  43. [51]

    Localized semantic feature mixers for effi- cient pedestrian detection in autonomous driving

    Abdul Hannan Khan, Mohammed Shariq Nawaz, and An- dreas Dengel. Localized semantic feature mixers for effi- cient pedestrian detection in autonomous driving. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5476–5485, 2023. 1

    2023

  44. [52]

    Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling

    Dahyun Kim, Chanjun Park, Sanghoon Kim, Wonsung Lee, Wonho Song, Yunsu Kim, Hyeonwoo Kim, Yungi Kim, Hyeonju Lee, Jihoo Kim, et al. Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling. arXiv preprint arXiv:2312.15166, 2023. 1

    arXiv

  45. [53]

    Densenets reloaded: Paradigm shift beyond resnets and vits

    Donghyun Kim, Byeongho Heo, and Dongyoon Han. Densenets reloaded: Paradigm shift beyond resnets and vits. arXiv preprint arXiv:2403.19588, 2024. 2

    2024 arXiv

  46. [54]

    A mechanistic interpretation of syllogistic reasoning in auto-regressive language models

    Geonhee Kim, Marco Valentino, and Andr ´e Freitas. A mechanistic interpretation of syllogistic reasoning in auto-regressive language models. arXiv preprint arXiv:2408.08590, 2024. 1

    2024 arXiv

  47. [55]

    Feature separation and recalibration for adversarial robustness

    Woo Jae Kim, Yoonki Cho, Junsik Jung, and Sung-Eui Yoon. Feature separation and recalibration for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8183– 8192, 2023. 1

    2023

  48. [56]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 4

    2014 arXiv

  49. [57]

    Mean-shift feature transformer

    Takumi Kobayashi. Mean-shift feature transformer. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6047–6056, 2024. 1

    2024

  50. [58]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 5

    2009

  51. [59]

    Gradient-based learning applied to document recognition

    Yann LeCun, L ´eon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324,

  52. [60]

    Fix the noise: Disen- tangling source feature for controllable domain translation

    Dongyeun Lee, Jae Young Lee, Doyeon Kim, Jaehyun Choi, Jaejun Yoo, and Junmo Kim. Fix the noise: Disen- tangling source feature for controllable domain translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14224–14234, 2023. 1

    2023

  53. [61]

    Automated progres- sive learning for efficient training of vision transformers

    Changlin Li, Bohan Zhuang, Guangrun Wang, Xiaodan Liang, Xiaojun Chang, and Yi Yang. Automated progres- sive learning for efficient training of vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 12486–12496, 2022. 1

    2022

  54. [62]

    Robustness preserving fine-tuning using neuron importance

    Guangrui Li, Rahul Duggal, Aaditya Singh, Kaustav Kundu, Bing Shuai, and Jon Wu. Robustness preserving fine-tuning using neuron importance. 2024. 1

    2024

  55. [63]

    Formality is favored: Unraveling the learning preferences of large language models on data with conflicting knowl- edge, 2024

    Jiahuan Li, Yiqing Cao, Shujian Huang, and Jiajun Chen. Formality is favored: Unraveling the learning preferences of large language models on data with conflicting knowl- edge, 2024. 1

    2024

  56. [64]

    Adver- sarial feature hallucination networks for few-shot learning

    Kai Li, Yulun Zhang, Kunpeng Li, and Yun Fu. Adver- sarial feature hallucination networks for few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13470–13479, 2020. 1

    2020

  57. [65]

    Parameter-efficient fine- tuning in spectral domain for point cloud learning, 2024

    Dingkang Liang, Tianrui Feng, Xin Zhou, Yumeng Zhang, Zhikang Zou, and Xiang Bai. Parameter-efficient fine- tuning in spectral domain for point cloud learning, 2024. 1

    2024

  58. [66]

    Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning

    Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Ser- ena Yeung, and James Y Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. Advances in Neural Information Processing Sys- tems, 35:17612–17625, 2022. 1

    2022

  59. [67]

    Class incremental learning via likelihood ratio based task prediction

    Haowei Lin, Yijia Shao, Weinan Qian, Ningxin Pan, Yiduo Guo, and Bing Liu. Class incremental learning via likelihood ratio based task prediction. arXiv preprint arXiv:2309.15048, 2023. 1

    2023 arXiv

  60. [68]

    Zero-to- strong generalization: Eliciting strong capabilities of large language models iteratively without gold labels

    Chaoqun Liu, Qin Chao, Wenxuan Zhang, Xiaobao Wu, Boyang Li, Anh Tuan Luu, and Lidong Bing. Zero-to- strong generalization: Eliciting strong capabilities of large language models iteratively without gold labels. arXiv preprint arXiv:2409.12425, 2024. 1

    2024 arXiv

  61. [69]

    Soft augmentation for image classification

    Yang Liu, Shen Yan, Laura Leal-Taix ´e, James Hays, and Deva Ramanan. Soft augmentation for image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16241–16250, 2023. 1

    2023

  62. [70]

    Pat: Pruning- aware tuning for large language models

    Yijiang Liu, Huanrui Yang, Youxin Chen, Rongyu Zhang, Miao Wang, Yuan Du, and Li Du. Pat: Pruning- aware tuning for large language models. arXiv preprint arXiv:2408.14721, 2024. 1

    2024 arXiv

  63. [71]

    Unlearn efficient removal of knowledge in large language models

    Tyler Lizzo and Larry Heck. Unlearn efficient removal of knowledge in large language models. arXiv preprint arXiv:2408.04140, 2024. 1

    2024 arXiv

  64. [72]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 4

    2017 arXiv

  65. [73]

    Mono-internvl: Pushing the boundaries of monolithic multimodal large language mod- els with endogenous visual pre-training, 2024

    Gen Luo, Xue Yang, Wenhan Dou, Zhaokai Wang, Jifeng Dai, Yu Qiao, and Xizhou Zhu. Mono-internvl: Pushing the boundaries of monolithic multimodal large language mod- els with endogenous visual pre-training, 2024. 1

    2024

  66. [74]

    Language models” grok” to copy

    Ang Lv, Ruobing Xie, Xingwu Sun, Zhanhui Kang, and Rui Yan. Language models” grok” to copy. arXiv preprint arXiv:2409.09281, 2024. 1

    2024 arXiv

  67. [75]

    Curvature-balanced feature manifold learning for long-tailed classification

    Yanbiao Ma, Licheng Jiao, Fang Liu, Shuyuan Yang, Xu Liu, and Lingling Li. Curvature-balanced feature manifold learning for long-tailed classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15824–15835, 2023. 1

    2023

  68. [76]

    Finite scalar quantization: Vq-vae made simple

    Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: Vq-vae made simple. arXiv preprint arXiv:2309.15505, 2023. 1

    2023 arXiv

  69. [77]

    Pace: marrying generalization in parameter-efficient fine-tuning with con- sistency regularization

    Yao Ni, Shan Zhang, and Piotr Koniusz. Pace: marrying generalization in parameter-efficient fine-tuning with con- sistency regularization. arXiv preprint arXiv:2409.17137 ,

  70. [78]

    House of cards: Massive weights in llms

    Jaehoon Oh, Seungjun Shin, and Dokwan Oh. House of cards: Massive weights in llms. arXiv preprint arXiv:2410.01866, 2024. 1

    2024 arXiv

  71. [79]

    Zoom in: An in- troduction to circuits

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An in- troduction to circuits. Distill, 5(3):e00024–001, 2020. 1

    2020

  72. [80]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shya- mal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jef...

    2024

  73. [81]

    Dinov2: Learning robust visual features without supervi- sion

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervi- sion. arXiv preprint arXiv:2304.07193, 2023. 2

    2023 arXiv

  74. [82]

    Diverse feature learning by self-distillation and reset

    Sejik Park. Diverse feature learning by self-distillation and reset. arXiv preprint arXiv:2403.19941, 2024. 1

    2024 arXiv

  75. [83]

    Learning more generalized experts by merging experts in mixture-of-experts

    Sejik Park. Learning more generalized experts by merging experts in mixture-of-experts. arXiv preprint arXiv:2405.11530, 2024. 1

    2024 arXiv

  76. [84]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023. 1

    2023

  77. [85]

    Enhancing deformable local features by jointly learning to detect and describe keypoints

    Guilherme Potje, Felipe Cadar, Andr ´e Araujo, Renato Mar- tins, and Erickson R Nascimento. Enhancing deformable local features by jointly learning to detect and describe keypoints. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 130...

    2023

  78. [86]

    Cafeboost: Causal feature boost to eliminate task-induced bias for class incremental learning

    Benliu Qiu, Hongliang Li, Haitao Wen, Heqian Qiu, Lanx- iao Wang, Fanman Meng, Qingbo Wu, and Lili Pan. Cafeboost: Causal feature boost to eliminate task-induced bias for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2023

  79. [87]

    Unlocking emergent modularity in large language models

    Zihan Qiu, Zeyu Huang, and Jie Fu. Unlocking emergent modularity in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the As- sociation for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2638–2660,

    2024

  80. [88]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recog- nition Challenge. International Journal of Computer Vision (I...

    2015

  81. [89]

    Lare: Latent augmentation us- ing regional embedding with vision-language model

    Kosuke Sakurai, Tatsuya Ishii, Ryotaro Shimizu, Linxin Song, and Masayuki Goto. Lare: Latent augmentation us- ing regional embedding with vision-language model. arXiv preprint arXiv:2409.12597, 2024. 1

    2024 arXiv

  82. [90]

    Scaling smart: Accelerating large language model pre-training with small model initial- ization

    Mohammad Samragh, Iman Mirzadeh, Keivan Alizadeh Vahid, Fartash Faghri, Minsik Cho, Moin Nabi, Devang Naik, and Mehrdad Farajtabar. Scaling smart: Accelerating large language model pre-training with small model initial- ization. arXiv preprint arXiv:2409.12903, 2024. 1

    2024 arXiv

  83. [91]

    Overcoming catastrophic forgetting with hard attention to the task

    Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference on ma- chine learning, pages 4548–4557. PMLR, 2018

    2018

  84. [92]

    Adaptive subspaces for few-shot learn- ing

    Christian Simon, Piotr Koniusz, Richard Nock, and Mehrtash Harandi. Adaptive subspaces for few-shot learn- ing. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 4136–4145, 2020

    2020

  85. [93]

    Sg- mim: Structured knowledge guided efficient pre-training for dense prediction

    Sumin Son, Hyesong Choi, and Dongbo Min. Sg- mim: Structured knowledge guided efficient pre-training for dense prediction. arXiv preprint arXiv:2409.02513 ,

    arXiv

  86. [94]

    Out- of-distribution generalization via composition: a lens through induction heads in transformers

    Jiajun Song, Zhuoyan Xu, and Yiqiao Zhong. Out- of-distribution generalization via composition: a lens through induction heads in transformers. arXiv preprint arXiv:2408.09503, 2024. 1

    2024 arXiv

  87. [95]

    Striving for simplicity: The all convolutional net

    Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806 ,

    arXiv

  88. [96]

    Dropout: a simple way to prevent neural networks from overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The jour- nal of machine learning research, 15(1):1929–1958, 2014. 2

    1929

  89. [97]

    Locat- ing information in large language models via random ma- trix theory

    Max Staats, Matthias Thamm, and Bernd Rosenow. Locat- ing information in large language models via random ma- trix theory. arXiv preprint arXiv:2410.17770, 2024. 1

    2024

  90. [98]

    Svfit: Parameter- efficient fine-tuning of large pre-trained models using sin- gular values

    Chengwei Sun, Jiwei Wei, Yujia Wu, Yiming Shi, Shiyuan He, Zeyu Ma, Ning Xie, and Yang Yang. Svfit: Parameter- efficient fine-tuning of large pre-trained models using sin- gular values. arXiv preprint arXiv:2409.05926, 2024. 1

    2024 arXiv

  91. [99]

    Smile: Zero-shot sparse mixture of low-rank experts construction from pre-trained foundation models

    Anke Tang, Li Shen, Yong Luo, Shuai Xie, Han Hu, Lefei Zhang, Bo Du, and Dacheng Tao. Smile: Zero-shot sparse mixture of low-rank experts construction from pre-trained foundation models. arXiv preprint arXiv:2408.10174 ,

    arXiv

  92. [100]

    Scaling monosemanticity: Extracting in- terpretable features from claude 3 sonnet

    Adly Templeton. Scaling monosemanticity: Extracting in- terpretable features from claude 3 sonnet. Anthropic, 2024. 2, 1

    2024

  93. [101]

    Deit iii: Revenge of the vit

    Hugo Touvron, Matthieu Cord, and Herv ´e J´egou. Deit iii: Revenge of the vit. In European conference on computer vision, pages 516–533. Springer, 2022. 1

    2022

  94. [102]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. 1

    2023 arXiv

  95. [103]

    Preserving generalization of language models in few-shot continual relation extraction

    Quyen Tran, Nguyen Xuan Thanh, Nguyen Hoang Anh, Nam Le Hai, Trung Le, Linh Van Ngo, and Thien Huu Nguyen. Preserving generalization of language models in few-shot continual relation extraction. arXiv preprint arXiv:2410.00334, 2024. 1

    arXiv

  96. [104]

    Regularizing meta- learning via gradient dropout

    Hung-Yu Tseng, Yi-Wen Chen, Yi-Hsuan Tsai, Sifei Liu, Yen-Yu Lin, and Ming-Hsuan Yang. Regularizing meta- learning via gradient dropout. In Proceedings of the Asian Conference on Computer Vision, 2020. 1

    2020

  97. [105]

    Which pro- gramming language and what features at pre-training stage affect downstream logical inference performance?, 2024

    Fumiya Uchiyama, Takeshi Kojima, Andrew Gambardella, Qi Cao, Yusuke Iwasawa, and Yutaka Matsuo. Which pro- gramming language and what features at pre-training stage affect downstream logical inference performance?, 2024. 1

    2024

  98. [106]

    Hvclip: High-dimensional vector in clip for unsupervised domain adaptation

    Sol Vesdapunt, Kah Kuen Fu, Yue Rex Wu, Xu Zhang, and Pradeep Natarajan. Hvclip: High-dimensional vector in clip for unsupervised domain adaptation. 2024. 1

    2024

  99. [107]

    Learning bottleneck concepts in image classifi- cation

    Bowen Wang, Liangzhi Li, Yuta Nakashima, and Hajime Nagahara. Learning bottleneck concepts in image classifi- cation. In Proceedings of the ieee/cvf conference on com- puter vision and pattern recognition , pages 10962–10971,

  100. [108]

    Image cropping with spatial-aware feature and rank consistency

    Chao Wang, Li Niu, Bo Zhang, and Liqing Zhang. Image cropping with spatial-aware feature and rank consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10052–10061, 2023. 1

    2023

  101. [109]

    Yolov9: Learning what you want to learn us- ing programmable gradient information

    Chien-Yao Wang, I-Hau Yeh, and Hong-Yuan Mark Liao. Yolov9: Learning what you want to learn us- ing programmable gradient information. arXiv preprint arXiv:2402.13616, 2024. 4, 5, 1

    2024 arXiv

  102. [110]

    Efficienttrain: Exploring gen- eralized curriculum learning for training visual backbones

    Yulin Wang, Yang Yue, Rui Lu, Tianjiao Liu, Zhao Zhong, Shiji Song, and Gao Huang. Efficienttrain: Exploring gen- eralized curriculum learning for training visual backbones. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5852–5864, 2023. 1

    2023

  103. [111]

    A case where a spindly two-layer linear network whips any neural network with a fully connected input layer

    Manfred K Warmuth, Wojciech Kotłowski, and Ehsan Amid. A case where a spindly two-layer linear network whips any neural network with a fully connected input layer. arXiv preprint arXiv:2010.08625, 2020. 1

    2010 arXiv

  104. [112]

    Pytorch image models

    Ross Wightman. Pytorch image models. https : / / github . com / rwightman / pytorch - image - models, 2019. 2, 5

    2019

  105. [113]

    Panadapter: Two-stage fine-tuning with spatial-spectral priors injecting for pansharpening

    RuoCheng Wu, ZiEn Zhang, ShangQi Deng, YuLe Duan, and LiangJian Deng. Panadapter: Two-stage fine-tuning with spatial-spectral priors injecting for pansharpening. arXiv preprint arXiv:2409.06980, 2024. 1

    2024 arXiv

  106. [114]

    Vit-comer: Vi- sion transformer with convolutional multi-scale feature in- teraction for dense predictions

    C Xia, X Wang, F Lv, X Hao, and Y Shi. Vit-comer: Vi- sion transformer with convolutional multi-scale feature in- teraction for dense predictions. arxiv 2024. arXiv preprint arXiv:2403.07392. 1

    2024 arXiv

  107. [115]

    Improving transformers with dynamically composable multi-head attention

    Da Xiao, Qingye Meng, Shengping Li, and Xingyuan Yuan. Improving transformers with dynamically composable multi-head attention. arXiv preprint arXiv:2405.08553 , 2024

    2024 arXiv

  108. [116]

    Making small language models better multi-task learners with mixture-of-task-adapters

    Yukang Xie, Chengyu Wang, Junbing Yan, Jiyong Zhou, Feiqi Deng, and Jun Huang. Making small language models better multi-task learners with mixture-of-task-adapters. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 1094–1097, 2024. 1

    2024

  109. [117]

    Collapsed language models promote fairness,

    Jingxuan Xu, Wuyang Chen, Linyi Li, Yao Zhao, and Yun- chao Wei. Collapsed language models promote fairness,

  110. [118]

    Dynamic prompt learning: Addressing cross- attention leakage for text-based image editing

    Fei Yang, Shiqi Yang, Muhammad Atif Butt, Joost van de Weijer, et al. Dynamic prompt learning: Addressing cross- attention leakage for text-based image editing. Advances in Neural Information Processing Systems, 36:26291–26303,

  111. [119]

    Graphlora: Structure-aware contrastive low-rank adap- tation for cross-graph transfer learning

    Zhe-Rui Yang, Jindong Han, Chang-Dong Wang, and Hao Liu. Graphlora: Structure-aware contrastive low-rank adap- tation for cross-graph transfer learning. arXiv preprint arXiv:2409.16670, 2024. 1

    2024 arXiv

  112. [120]

    Adaptive spot-guided transformer for consistent local feature matching

    Jiahuan Yu, Jiahao Chang, Jianfeng He, Tianzhu Zhang, Jiyang Yu, and Feng Wu. Adaptive spot-guided transformer for consistent local feature matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21898–21908, 2023. 1

    2023

  113. [121]

    Block selection method for using fea- ture norm in out-of-distribution detection

    Yeonguk Yu, Sungho Shin, Seongju Lee, Changhyun Jun, and Kyoobin Lee. Block selection method for using fea- ture norm in out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 15701–15711, 2023. 1

    2023

  114. [122]

    Cutmix: Regu- larization strategy to train strong classifiers with localizable features

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regu- larization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023–6032, 2019. 1

    2019

  115. [123]

    Visualizing and understanding convolutional networks

    MD Zeiler. Visualizing and understanding convolutional networks. In European conference on computer vi- sion/arXiv, 2014. 1

    2014

  116. [124]

    Feature representation learning with adaptive displacement generation and trans- former fusion for micro-expression recognition

    Zhijun Zhai, Jianhui Zhao, Chengjiang Long, Wenju Xu, Shuangjiang He, and Huijuan Zhao. Feature representation learning with adaptive displacement generation and trans- former fusion for micro-expression recognition. In Pro- ceedings of the IEEE/CVF Conference on Computer Visi...

    2023

  117. [125]

    Deepemd: Few-shot image classification with differen- tiable earth mover’s distance and structured classifiers

    Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classification with differen- tiable earth mover’s distance and structured classifiers. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 12203–12213, 2020. 1

    2020

  118. [126]

    mixup: Beyond empirical risk minimiza- tion

    Hongyi Zhang. mixup: Beyond empirical risk minimiza- tion. arXiv preprint arXiv:1710.09412, 2017. 1

    2017 arXiv

  119. [127]

    Co-occurrence is not factual association in language models

    Xiao Zhang, Miao Li, and Ji Wu. Co-occurrence is not factual association in language models. arXiv preprint arXiv:2409.14057, 2024. 1

    2024 arXiv

  120. [128]

    Con- nect, collapse, corrupt: Learning cross-modal tasks with uni-modal data

    Yuhui Zhang, Elaine Sui, and Serena Yeung-Levy. Con- nect, collapse, corrupt: Learning cross-modal tasks with uni-modal data. arXiv preprint arXiv:2401.08567 , 2024. 1

    2024 arXiv

  121. [129]

    Enhancing cross- domain pre-trained decision transformers with adaptive at- tention

    Wenhao Zhao, Qiushui Xu, Linjie Xu, Lei Song, Jinyu Wang, Chunlai Zhou, and Jiang Bian. Enhancing cross- domain pre-trained decision transformers with adaptive at- tention. arXiv preprint arXiv:2409.06985, 2024. 1

    2024 arXiv

  122. [130]

    Learning to obstruct few-shot image classification over re- stricted classes

    Amber Yijia Zheng, Chiao-An Yang, and Raymond A Yeh. Learning to obstruct few-shot image classification over re- stricted classes. arXiv preprint arXiv:2409.19210, 2024. 1

    2024 arXiv

  123. [131]

    Learning data augmen- tation strategies for object detection

    Barret Zoph, Ekin D Cubuk, Golnaz Ghiasi, Tsung-Yi Lin, Jonathon Shlens, and Quoc V Le. Learning data augmen- tation strategies for object detection. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part XXVII 16, pages 566–

    2020

  124. [133]

    Acknowledgments We appreciate the high-performance GPU computing sup- port of HPC-AI Open Infrastructure via GIST SCENT

    Supplementary 6.1. Acknowledgments We appreciate the high-performance GPU computing sup- port of HPC-AI Open Infrastructure via GIST SCENT. 6.2. Related Work: Feature Analysis There are several methods to analyze features: directly an- alyzing features, analyzing the relations...

  125. [583]

    1 ResidualDroppath: Enhancing Feature Reuse over Residual Connections Supplementary Material

    Springer, 2020. 1 ResidualDroppath: Enhancing Feature Reuse over Residual Connections Supplementary Material

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.