Pith. sign in

REVIEW 4 major objections 6 minor 83 references

Vision Transformer Neural Architecture Search for Out-of-Distribution Generalization: Benchmark and Insights

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper's central claim is that for Vision Transformers, architecture design—especially embedding dimension—drives out-of-distribution generalization as much as training-time robustness methods, and that standard in-distribution…

desk verdict A genuinely useful benchmark resource whose central scientific claims rest on an unvalidated weight-sharing assumption; worth serious refereeing, but with major revision required. read the letter →

arxiv 2501.03782 v1 pith:PXOT7PPQ submitted 2025-01-07 cs.LG

Sy-Tuyen Ho , Tuan Van Vo , Somayeh Ebrahimkhani , Ngai-Man Cheung This is my paper · ORCID
classification cs.LG
keywords VisionTransformerout-of-distributiongeneralizationneuralarchitecturesearchtraining-freeNASzero-costproxyembeddingdimensionAutoFormerhigh-frequencycomponents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper builds OoD-ViT-NAS, a benchmark of 3,000 Vision Transformer architectures sampled from the AutoFormer search space and scored on eight out-of-distribution (OoD) image datasets. It uses this benchmark to argue that ViT architecture choice has a large effect on OoD generalization, larger in some cases than a state-of-the-art training-time OoD method, and that in-distribution (ID) accuracy is a misleading guide to which architectures will generalize. It further claims that existing training-free neural architecture search (NAS) proxies predict ID accuracy better than OoD accuracy, and that simple parameter or FLOP counts beat all of them at predicting OoD performance. The paper's positive design insight is that increasing the embedding dimension of a ViT generally improves OoD accuracy, and it offers evidence that this happens because larger embedding dimensions help the model learn high-frequency image components.

What carries the argument

The machinery is one-shot NAS weight sharing: three AutoFormer supernets (Tiny, Small, and Base) are trained once, and 3,000 subnets are extracted by letting them inherit the supernet weights, which makes large-scale OoD evaluation computationally feasible. The analysis tools are Kendall's $\tau$ rank correlation for ID-versus-OoD and proxy-versus-OoD comparisons, plus a frequency-filtering experiment that removes high-frequency components at radius $r$ and measures the accuracy change to test whether larger embedding dimensions improve OoD by learning more high-frequency components. This machinery is load-bearing because every benchmark insight is a range or ranking computed on weight-sharing subnets.

What would settle it

Take a random sample of 30 architectures from the benchmark, train each one from scratch under the standard ImageNet protocol, and measure OoD accuracy on ImageNet-R and ImageNet-C; if their rankings correlate weakly with the weight-sharing subnet scores, or if the embedding-dimension advantage reverses, the paper's central insights fail.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that OoD generalization in ViTs is much more architecture-dependent than previous studies of a handful of human-designed models could show, and that the dependency has a clear direction: among AutoFormer's searchable attributes, embedding dimension is the strongest and most consistent lever, with an ablation correlation around Kendall's $\tau = 0.65$ when other attributes are held fixed. ID accuracy correlates only weakly with OoD accuracy on most shifts, and the ID-Pareto architectures are usually not the OoD-Pareto architectures. Nine training-free NAS scores, including recent ViT-specific ones, rank architectures worse than #Params or #FLOPs for OoD (best $\tau \approx 0.36$ for #Params versus $0.33$ for AutoProx), and all proxies fail on ImageNet-D. The paper frames this as evidence that architecture design itself should be a first-class object of OoD research, not a proxy for training methods.

Load-bearing premise

The load-bearing premise is that a subnetwork that inherits weights from a shared supernet behaves, for out-of-distribution accuracy, like the same architecture trained on its own; the paper only cites evidence for this equivalence on in-distribution accuracy.

Editorial extensions

If this is right

  • If the claims hold, NAS for ViTs should benchmark OoD accuracy directly rather than rely on ID accuracy, because the two rankings diverge.
  • New training-free NAS proxies for ViT OoD accuracy must beat #Params and #FLOPs; the paper sets those simple counts as the bar.
  • Architects can improve OoD generalization by widening the embedding dimension, potentially more cheaply than compound scaling of depth, heads, and MLP ratio.
  • The benchmark's 3,000 scored architectures across eight shift types enable future studies of which components drive robustness on specific corruption or natural shift types.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the benchmark uses weight-sharing subnets, its rankings may not fully transfer to standalone-trained ViTs; the paper's cited evidence for equivalence concerns ID accuracy only, so the embedding-dimension rule should be re-tested with independent training before being treated as a design law. This is an extension of the paper's own caveat, not a claim it tests.
  • The high-frequency-component explanation suggests a testable intervention: if larger embedding dimensions help by preserving high-frequency content, then high-frequency-emphasizing augmentation during training might reproduce part of the OoD gain at constant architecture size, something the paper does not try.
  • The uniform failure of all proxies on ImageNet-D hints that diffusion-generated OoD shifts may need a fundamentally different architectural response than corruption or natural shifts, an avenue the paper leaves open.
  • The finding that OoD accuracy spread widens with shift severity implies that architecture design will matter more as deployment shifts get harder, so the 11.85% spread figure is likely a lower bound for harder future benchmarks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces OoD-ViT-NAS, a benchmark of 3,000 ViT architectures sampled from the AutoFormer one-shot supernets (Tiny/Small/Base) and evaluated on 8 ImageNet-scale OoD datasets (IN-C, IN-P, IN-A, IN-O, IN-R, IN-Sketch, Stylized-IN, IN-D). Using this benchmark, the authors investigate (i) the range of OoD accuracy across architectures, (ii) the correlation between ID and OoD accuracy, (iii) the ability of nine training-free NAS proxies to predict OoD accuracy versus simple #Param/#Flops, and (iv) the impact of ViT architectural attributes, concluding that embedding dimension is the most important attribute and that increasing it generally improves OoD generalization. The authors also present a frequency-based explanation and a small human-designed ViT study in the appendix to support the embedding-dimension finding.

Significance. If the benchmark's ranking of architectures is trustworthy, this would be a valuable community resource: it is the first large-scale ViT NAS benchmark targeting OoD generalization, and the released data/code are likely to catalyze future work. The paper also makes concrete, falsifiable claims about the limits of training-free NAS proxies for OoD prediction and about the importance of embedding dimension. However, the benchmark's reliance on weight-sharing supernet subnets, without validation that OoD rankings transfer to independently trained ViTs, places a major asterisk on every insight. The statistical evidence for the training-free NAS comparison is also weaker than the text suggests. The human-designed ViT experiment (Appx 8) provides some partial support for the embedding-dimension conclusion, but it is on a different scale and does not cover the full benchmark.

major comments (4)
  1. [Sec. 3, Protocol] The benchmark evaluates only weight-sharing subnets extracted from AutoFormer supernets; the paper justifies this by citing [20, 6] for the claim that subnet performance is comparable to independently trained architectures. Those citations support ID accuracy only. No experiment in the paper validates that the OoD accuracy ranking of subnets matches the ranking of the same architectures trained from scratch. This is load-bearing because every downstream analysis (the 11.85% OoD spread, the ID-OoD decoupling in Sec. 4.2, the training-free NAS comparison in Sec. 4.3, and the embedding-dimension claim in Sec. 4.4) is computed from these subnet evaluations. If OoD behavior is more sensitive to weight-sharing artifacts than ID behavior, the conclusions could be artifacts. The authors should either (a) train a representative subset of the 3,000 architectures independently and show that the OoD rankings correlate with the subnet rankings, or (b) explicitly reframe all claims as being about one-shot supernet subnets rather than ViT architectures in general. The current wording in the Abstract and Sec. 1 overstates the scope.
  2. [Sec. 4.3, Table 2] The claim that simple proxies like #Param and #Flops 'surprisingly outperform' complex training-free NAS is not justified by the reported statistics. The Kendall tau values are presented as means with standard deviations (e.g., #Param 0.3600±0.2321 vs AutoProx-A 0.3303±0.2384; #Flops 0.3537±0.2327 vs DSS 0.3421±0.2365). These differences are far smaller than the quoted standard deviations and are not tested for significance. Given the large uncertainty, the paper cannot support the superlative conclusion in the Abstract and Sec. 4.3. The authors should run a paired significance test (e.g., bootstrap over search spaces/datasets) and report confidence intervals, or temper the claim to a descriptive observation.
  3. [Sec. 4.1 and Table 3] The comparison of the OoD accuracy range (e.g., 3.80%/4.86%/2.74%) with a 1.9% improvement from the SOTA training method [30] is not apples-to-apples: the former is a spread between the best and worst architecture in the search space, not an improvement over a baseline achieved by a method. Additionally, Table 3 compares an architecture trained on IN-100 (labelled 'Ours') with ViT-B-32 and ViT-L-32 without stating that the baselines were evaluated under the same training protocol; the table likely mixes IN-100-trained results with ImageNet-scale reference numbers. This makes the claim that the embedding-dimension insight yields architectures that 'outperform' well-established human-designed ViTs unsupported. The authors need to clarify the training/evaluation protocol for all models in Table 3 and, if the baselines were not trained identically, remove the direct comparison.
  4. [Sec. 4.4 and Appx 17.1] The ablation in Appx 17.1 claims to fix all other architectural attributes when varying embedding dimension, but it does so by restricting to architectures with mean #Head = 6±0.05 and mean MLP_Ratio = 3.5±0.05. This is not a controlled ablation because the per-layer values of heads and MLP ratio still vary within the small range, and the selection of architectures by these means may introduce confounding with other unobserved variables. The layer-wise analysis (Appx 17.2) is a useful complement, and the human-designed ViT experiment (Appx 8) provides partial independent evidence, but the main-benchmark correlation in Fig. 1-c is computed over all 3,000 architectures without such controls. The causal language 'increasing embedding dimensions generally enhances performance' should be softened to 'is positively associated with OoD accuracy in this benchmark' unless a rigorous controlled study is added.
minor comments (6)
  1. [Abstract and Sec. 1] The phrase 'with up to 11.85% improvement for some OoD shifts' is ambiguous: is this a range between two architectures, an improvement over a baseline, or something else? Please clarify in the text and define the quantity precisely when it is first used.
  2. [Sec. 4.1] There is a duplicated phrase: 'analyzing 1,000 architectures from the Autoformer-Small search space within our OoD-ViT-NAS benchmark for1, 000 architectures in Autoformer-Small search space within our benchmark on IN-C.' The sentence should be rewritten.
  3. [Appx 10.1, Table 10.2] The rows for Supernet-Tiny appear to have Max and Min reversed (Max 192 vs Min 240), and the stated ranges with step sizes are inconsistent with typical AutoFormer settings. Please check and correct the table.
  4. [Sec. 4.3, Fig. 1-b] The claim that all training-free methods 'consistently fail to predict IN-D performance' is based on near-zero negative correlations (e.g., -0.03 to -0.09 in Appx Table 15.5). This is better described as a lack of predictive power rather than 'failure'; negative values may also indicate a peculiarity of the IN-D dataset that deserves discussion.
  5. [Sec. 4.4] The statement 'Our comprehensive OoD-ViT-NAS benchmark sheds light on a previously unknown relationship' is too strong; several prior works (e.g., [13,15]) have noted dependencies between ViT capacity and robustness. Please soften the novelty claim.
  6. [Throughout] The manuscript contains many typos and awkward phrases (e.g., 'MLP_Ration', 'MLP_Tatio', 'ration' for 'ratio', inconsistent notation in Appx 11, and duplicated sentences in Sec. 4.1). A thorough proofreading pass is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: OoD-ViT-NAS is an external empirical benchmark, and the main debatable assumption (weight-sharing subnet evaluation) is a validity concern, not a derivation that reduces to its own inputs.

full rationale

OoD-ViT-NAS is an empirical benchmark: all 3,000 entries are measured on external datasets (IN-C, IN-A, IN-O, IN-P, IN-D, IN-R, IN-Sketch, Stylized-IN), not derived from the paper's own equations. The central claims—architecture affects OoD accuracy, ID-OoD decoupling, training-free NAS weakness, #Param/#Flop strength, and the embedding-dimension benefit—are correlational summaries of these external measurements. The only load-bearing methodological choice is the evaluation of weight-sharing subnets sampled from AutoFormer supernets (Sec. 3 Protocol). The paper justifies this by citing [20,6] (BigNAS and AutoFormer), which are prior external works by other authors; this is not a self-citation chain. Whether subnet OoD rankings transfer to independently trained ViTs is a validity/transfer concern, not circularity: even if the rankings were artifacts of weight sharing, the paper's claims would be false, but they would not be true by definition. The embedding-dimension insight is first observed in-sample on the 3,000-subnet benchmark, but Appx. 8 independently trains human-designed ViT configurations on IN-100 and evaluates on IN-R, providing separate evidence that partially checks the transfer concern. No fitted parameter is renamed as a prediction, no conclusion is assumed in its own premise, and no equation reduces the target result to an input. Therefore no specific circular step can be exhibited, and the honest finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The benchmark itself is a new artifact but not an invented entity in the physics sense. The central assumptions are the transferability of one-shot subnet performance to standalone models, and the representativeness of the AutoFormer search space. No new particles, forces, or conserved quantities are introduced.

free parameters (1)
  • Ablation attribute values (mean #Head = 6 ± 0.05, mean MLP_Ratio = 3.5 ± 0.05, Embed_Dim = 384, Depth = 13) = 6 ± 0.05 heads, 3.5 ± 0.05 MLP ratio, 384 embedding dim, 13 depth
    These hand-chosen fixed values in Appx 17.1 define the ablation ranges for isolating each attribute; they shape the reported per-attribute Kendall correlations that support the embedding-dimension conclusion.
assumptions (3)
  • domain assumption Subnets sampled from AutoFormer supernets inherit weights whose performance is comparable to standalone training, even under OoD shifts.
    Invoked in Sec 3 Protocol, citing [20, 6]; the cited evidence is about ID accuracy, and no OoD validation is provided.
  • domain assumption The AutoFormer search space and its attribute ranges are representative enough to support general conclusions about ViT architecture-OoD relationships.
    Sec 3 Search Space defines all conclusions on this search space, with only one additional check on a human-designed search space in Appx 8.
  • domain assumption OoD accuracy and AUPR computed on the 8 chosen datasets capture the intended notion of OoD generalization.
    Sec 3 Metrics follows prior works, but several datasets (IN-A, IN-O) use only 200 classes, which can distort accuracy-based comparisons.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vision Transformer Neural Architecture Search for Out-of-Distribution Generalization: Benchmark and Insights." pith.science (2026). https://pith.science/paper/PXOT7PPQ

@misc{pith2026250103782,
  author       = {Pith},
  title        = {Pith review of: Vision Transformer Neural Architecture Search for Out-of-Distribution Generalization: Benchmark and Insights},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PXOT7PPQ}},
  note         = {Machine review of arXiv:2501.03782}
}
read the original abstract

While ViTs have achieved across machine learning tasks, deploying them in real-world scenarios faces a critical challenge: generalizing under OoD shifts. A crucial research gap exists in understanding how to design ViT architectures, both manually and automatically, for better OoD generalization. To this end, we introduce OoD-ViT-NAS, the first systematic benchmark for ViTs NAS focused on OoD generalization. This benchmark includes 3000 ViT architectures of varying computational budgets evaluated on 8 common OoD datasets. Using this benchmark, we analyze factors contributing to OoD generalization. Our findings reveal key insights. First, ViT architecture designs significantly affect OoD generalization. Second, ID accuracy is often a poor indicator of OoD accuracy, highlighting the risk of optimizing ViT architectures solely for ID performance. Third, we perform the first study of NAS for ViTs OoD robustness, analyzing 9 Training-free NAS methods. We find that existing Training-free NAS methods are largely ineffective in predicting OoD accuracy despite excelling at ID accuracy. Simple proxies like Param or Flop surprisingly outperform complex Training-free NAS methods in predicting OoD accuracy. Finally, we study how ViT architectural attributes impact OoD generalization and discover that increasing embedding dimensions generally enhances performance. Our benchmark shows that ViT architectures exhibit a wide range of OoD accuracy, with up to 11.85% improvement for some OoD shifts. This underscores the importance of studying ViT architecture design for OoD. We believe OoD-ViT-NAS can catalyze further research into how ViT designs influence OoD generalization.

Figures

Figures reproduced from arXiv: 2501.03782 by the authors.

Figure 1
Figure 1. We propose, OoD-ViT-NAS, the first comprehensive benchmark for NAS on OoD generalization of ViT architectures. Then, we comprehensively investigate OoD generalization for ViT. The detailed of 8 OoD datasets in our investigation can be found in Tab. 1. In this figure, we show the Kendall τ ranking correlation between OoD accuracy of different datasets on the left and different quantities at the bottom. Our analysis u… view at source ↗
Figure 2
Figure 2. Our analysis of the OoD accuracy range highlights the significant influence of ViT architectural designs on OoD accuracy. (Sec. 4.1) The numbers within each violin plot for each sub-figure (e.g., IN-D 9.79 (1.06), 9.65 (2.25), and 7.99 (0.56)) denote the corresponding OoD (ID) accuracy range of architectures sampled from Autoformer-Tiny/Small/Base search space, respectively. See Appx. 12 for additional plots and res… view at source ↗
Figure 3
Figure 3. Visualization of OoD accuracy range across OoD shift severity. We conduct the analysis on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (31 more)
Figure 4
Figure 4. Figure 4: Analysis of OoD Generalization Performance of Pareto Architectures for ID accuracy. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The effect of #Embed_Dim on robustness generalization of ViTs. The numbers denote the [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Following setting in [77, 80, 79], the ViTs which were trained on original ID data, are now tested on high frequency components (HFC) of OoD samples, with r as the radius for frequency filtering. The higher the OoD accuracy, the more HFC learned in the model. embedding…
Figure 10.1
Figure 10.1. Figure 10.1: Visualization of different OoD shifts across 8 datasets used to construct our OoD-ViT [PITH_FULL_IMAGE:figures/full_fig_p018_10_1.png]
Figure 10.2
Figure 10.2. Figure 10.2: Examples from ImageNet-D [26]. These examples are generated by Stable Diffusion [82] and only hard examples are kept. These examples could be distorted or unrealistic in object￾background placements. We visualize a few examples of OoD datasets in [PITH_FULL_IMAGE:f…
Figure 10.3
Figure 10.3. Figure 10.3: Visualization of different corruptions and 5 different OoD shift severity for ImageNet-C [PITH_FULL_IMAGE:figures/full_fig_p020_10_3.png]
Figure 11.4
Figure 11.4. Figure 11.4: The structure of the merged json file for our OoD-ViT-NAS benchmark for Autoformer￾Small search space. The structures of the merged json files for Autoformer-Tiny/Base are similars. 12 Additional results on the analysis of OoD accuracy range In the main paper, we vi…
Figure 12.5
Figure 12.5. Figure 12.5: As in Figure 2, our analysis on the OoD accuracy range highlights the significant [PITH_FULL_IMAGE:figures/full_fig_p023_12_5.png]
Figure 12.6
Figure 12.6. Figure 12.6: Visualization of OoD accuracy range across IN-C OoD shift severity. The experiments [PITH_FULL_IMAGE:figures/full_fig_p024_12_6.png]
Figure 13
Figure 13. Figure 13: , and Fig. 13.9, respectively [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]
Figure 13.7
Figure 13.7. Figure 13.7: Kendall τ rank correlation coefficient between ID and OoD accuracies computed on all 3000 architectures in our OoD-ViT-NAS benchmark. Measurements are computed on different corruptions of IN-C [PITH_FULL_IMAGE:figures/full_fig_p025_13_7.png]
Figure 13.8
Figure 13.8. Figure 13.8: Kendall τ rank correlation coefficient between ID and OoD accuracies computed on all 3000 architectures in our OoD-ViT-NAS benchmark. Measurements are computed on various IN-D OoD shifts. 14 Additional results on the analysis of OoD Performance of Pareto architectur…
Figure 13.9
Figure 13.9. Figure 13.9: Kendall τ rank correlation coefficient between ID and OoD accuracies computed on all 3000 architectures in our OoD-ViT-NAS benchmark. Measurements are computed on various IN-P OoD shifts. 15 Additional Results for Benchmarking Zero-cost Proxies In the main submissio…
Figure 13.10
Figure 13.10. Figure 13.10: Kendall rank correlation coefficient between ID and OoD performance computed on [PITH_FULL_IMAGE:figures/full_fig_p027_13_10.png]
Figure 17.11
Figure 17.11. Figure 17.11: Kendall’s τ rank correlation coefficient between varying Embed_Dim and OoD accuracy. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_17_11.png]
Figure 17.12
Figure 17.12. Figure 17.12: Kendall’s τ rank correlation coefficient between varying network depth and all OoD accuracy. For MLP_Ratio and #Heads, we observe that the rank correlation coefficients of MLP_Ratio and #Heads in [PITH_FULL_IMAGE:figures/full_fig_p030_17_12.png]
Figure 17.13
Figure 17.13. Figure 17.13: Kendall’s τ rank correlation coefficient between mean MLP ratio and all OoD accuracy. We fix Embed_Dim = 384, Depth = 13, and mean #Head = 6 ± 0.05 [PITH_FULL_IMAGE:figures/full_fig_p031_17_13.png]
Figure 17.14
Figure 17.14. Figure 17.14: Kendall’s τ rank correlation coefficient between the mean number of heads and all OoD accuracy. We fix Embed_Dim = 384, Depth = 13, and mean MLP_Ratio = 3.5 ± 0.05. 17.2 Layer-Wise Analysis The number of heads and MLP ratio vary across layers, which allows for sear…
Figure 18
Figure 18. Figure 18: depicts the nine configurations of #Head to analyze the impact of [PITH_FULL_IMAGE:figures/full_fig_p032_18.png]
Figure 17.15
Figure 17.15. Figure 17.15: A visualization on the effect of changing MLP ratio per layer to OoD accuracy in 108 [PITH_FULL_IMAGE:figures/full_fig_p032_17_15.png]
Figure 17.16
Figure 17.16. Figure 17.16: A visualization on the effect of changing #Heads per layer to OoD accuracy in 108 [PITH_FULL_IMAGE:figures/full_fig_p033_17_16.png]
Figure 18.17
Figure 18.17. Figure 18.17: The possible configurations of #Heads at [PITH_FULL_IMAGE:figures/full_fig_p034_18_17.png]
Figure 18.18
Figure 18.18. Figure 18.18: The possible configurations of MLP ratio at [PITH_FULL_IMAGE:figures/full_fig_p034_18_18.png]
Figure 18.19
Figure 18.19. Figure 18.19: As in Figure 4, we show that lower OoD accuracy can be obtained for the higher ID [PITH_FULL_IMAGE:figures/full_fig_p036_18_19.png]
Figure 18.20
Figure 18.20. Figure 18.20: Visualization of Pareto architectures in [PITH_FULL_IMAGE:figures/full_fig_p037_18_20.png]
Figure 18.21
Figure 18.21. Figure 18.21: Visualization of Pareto architectures in [PITH_FULL_IMAGE:figures/full_fig_p038_18_21.png]
Figure 18.22
Figure 18.22. Figure 18.22: Visualization of Pareto architectures in [PITH_FULL_IMAGE:figures/full_fig_p039_18_22.png]
Figure 18.23
Figure 18.23. Figure 18.23: Visualization of Pareto architectures in [PITH_FULL_IMAGE:figures/full_fig_p040_18_23.png]
Figure 18.24
Figure 18.24. Figure 18.24: As in Figure 4, we show that lower OoD accuracy can be obtained for the higher ID [PITH_FULL_IMAGE:figures/full_fig_p041_18_24.png]
Figure 18.25
Figure 18.25. Figure 18.25: As in Figure 5, we show the potential impact of embedding dimension (Embed_Dim) [PITH_FULL_IMAGE:figures/full_fig_p042_18_25.png]
Figure 18.26
Figure 18.26. Figure 18.26: As in Figure 5, we show the potential impact of embedding dimension (Embed_Dim) [PITH_FULL_IMAGE:figures/full_fig_p043_18_26.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

83 extracted references · 55 canonical work pages

  1. [30]

    HYPO: Hyperspherical out-of-distribution generalization

    Haoyue Bai, Yifei Ming, Julian Katz-Samuels, and Yixuan Li. HYPO: Hyperspherical out-of-distribution generalization. In The Twelfth International Conference on Learning Representations, 2024

    work page 2024

  2. [1]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  3. [2]

    Maxvit: Multi-axis vision transformer

    Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. Maxvit: Multi-axis vision transformer. In European conference on computer vision, pages 459–479. Springer, 2022

    2022

  4. [3]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021

    2021

  5. [4]

    Exploring plain vision transformer backbones for object detection

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. In European Conference on Computer Vision, pages 280–296. Springer, 2022

    2022

  6. [5]

    Segvit: Semantic segmentation with plain vision transformers.Advances in Neural Information Processing Systems, 35:4971– 4982, 2022

    Bowen Zhang, Zhi Tian, Quan Tang, Xiangxiang Chu, Xiaolin Wei, Chunhua Shen, et al. Segvit: Semantic segmentation with plain vision transformers.Advances in Neural Information Processing Systems, 35:4971– 4982, 2022

    work page 2022

  7. [6]

    Autoformer: Searching transformers for visual recognition

    Minghao Chen, Houwen Peng, Jianlong Fu, and Haibin Ling. Autoformer: Searching transformers for visual recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12270–12280, 2021

    work page 2021

  8. [7]

    Prenas: Preferred one-shot learning towards efficient neural architecture search

    Haibin Wang, Ce Ge, Hesen Chen, and Xiuyu Sun. Prenas: Preferred one-shot learning towards efficient neural architecture search. In International Conference on Machine Learning, pages 35642–35654. PMLR, 2023

    work page 2023

Show all 83 references
  1. [8]

    Vitas: Vision transformer architecture search

    Xiu Su, Shan You, Jiyang Xie, Mingkai Zheng, Fei Wang, Chen Qian, Changshui Zhang, Xiaogang Wang, and Chang Xu. Vitas: Vision transformer architecture search. In European Conference on Computer Vision, pages 139–157. Springer, 2022

    2022

  2. [9]

    Elasticvit: Conflict-aware supernet training for deploying fast vision transformer on diverse mobile devices

    Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, and Mao Yang. Elasticvit: Conflict-aware supernet training for deploying fast vision transformer on diverse mobile devices. In Proceedings of the IEEE/CVF International Confere...

    2023

  3. [10]

    Training-free transformer architecture search

    Qinqin Zhou, Kekai Sheng, Xiawu Zheng, Ke Li, Xing Sun, Yonghong Tian, Jie Chen, and Rongrong Ji. Training-free transformer architecture search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10894–10903, 2022

    2022

  4. [11]

    Training-free transformer architecture search with zero-cost proxy guided evolution

    Qinqin Zhou, Kekai Sheng, Xiawu Zheng, Ke Li, Yonghong Tian, Jie Chen, and Rongrong Ji. Training-free transformer architecture search with zero-cost proxy guided evolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  5. [12]

    Nasvit: Neural architecture search for efficient vision transformers with gradient conflict-aware supernet training

    Chengyue Gong and Dilin Wang. Nasvit: Neural architecture search for efficient vision transformers with gradient conflict-aware supernet training. ICLR Proceedings 2022, 2022

    2022

  6. [13]

    Understanding Robustness of Transformers for Image Classification

    Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and Andreas Veit. Understanding Robustness of Transformers for Image Classification. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10231–10241, oct 2021

    2021

  7. [14]

    Robustart: Benchmarking robustness on architecture design and training techniques

    Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xianglong Liu, Dawn Song, Alan Yuille, et al. Robustart: Benchmarking robustness on architecture design and training techniques. arXiv preprint arXiv:2109.05211, 2021

    2021 arXiv

  8. [15]

    Out of distribution performance of state of art vision model

    Salman Rahman and Wonkwon Lee. Out of distribution performance of state of art vision model. arXiv preprint arXiv:2301.10750, 2023

    2023 arXiv

  9. [16]

    Searching the search space of vision transformer

    Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, and Haibin Ling. Searching the search space of vision transformer. Advances in Neural Information Processing Systems, 34:8714–8726, 2021

    2021

  10. [17]

    Auto-prox: Training-free vision transformer architecture search via automatic proxy discovery

    Zimian Wei, Peijie Dong, Zheng Hui, Anggeng Li, Lujun Li, Menglong Lu, Hengyue Pan, and Dongsheng Li. Auto-prox: Training-free vision transformer architecture search via automatic proxy discovery. In Proceedings of the AAAI Conference on Artificial Intelligence, number 14, pag...

    2024

  11. [18]

    Hr-nas: Searching efficient high-resolution neural architectures with lightweight transformers

    Mingyu Ding, Xiaochen Lian, Linjie Yang, Peng Wang, Xiaojie Jin, Zhiwu Lu, and Ping Luo. Hr-nas: Searching efficient high-resolution neural architectures with lightweight transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages ...

    2021

  12. [19]

    Uninet: Unified architecture search with convolution, transformer, and mlp

    Jihao Liu, Xin Huang, Guanglu Song, Hongsheng Li, and Yu Liu. Uninet: Unified architecture search with convolution, transformer, and mlp. In European Conference on Computer Vision, pages 33–49. Springer, 2022

    2022

  13. [20]

    Bignas: Scaling up neural architecture search with big single-stage models

    Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas Huang, Xiaodan Song, Ruoming Pang, and Quoc Le. Bignas: Scaling up neural architecture search with big single-stage models. In Computer Vision–ECCV 2020: 16th European Conference...

    2020

  14. [21]

    All tokens matter: Token labeling for training better vision transformers

    Zi-Hang Jiang, Qibin Hou, Li Yuan, Daquan Zhou, Yujun Shi, Xiaojie Jin, Anran Wang, and Jiashi Feng. All tokens matter: Token labeling for training better vision transformers. Advances in neural information processing systems, 34:18590–18602, 2021

    2021

  15. [22]

    Crossvit: Cross-attention multi-scale vision transformer for image classification

    Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification. InProceedings of the IEEE/CVF international conference on computer vision, pages 357–366, 2021

    2021

  16. [23]

    Pre-trained image processing transformer

    Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. Pre-trained image processing transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12299–12310, 2021

    2021

  17. [24]

    Benchmarking neural network robustness to common corruptions and perturbations

    Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019

    1903 arXiv

  18. [25]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15262–15271, 2021

    2021

  19. [26]

    Imagenet-d: Benchmarking neural network robustness on diffusion synthetic object

    Chenshuang Zhang, Fei Pan, Junmo Kim, In So Kweon, and Chengzhi Mao. Imagenet-d: Benchmarking neural network robustness on diffusion synthetic object. arXiv preprint arXiv:2403.18775, 2024

    2024 arXiv

  20. [27]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF international c...

    2021

  21. [28]

    Learning robust global representations by penalizing local predictive power

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. Advances in Neural Information Processing Systems, 32, 2019

    2019

  22. [29]

    Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness

    Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231, 2018

    2018 arXiv

  23. [31]

    Invariant risk minimization

    Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019

    1907 arXiv

  24. [32]

    Domain generalization for object recognition with multi-task autoencoders

    Muhammad Ghifary, W Bastiaan Kleijn, Mengjie Zhang, and David Balduzzi. Domain generalization for object recognition with multi-task autoencoders. In Proceedings of the IEEE international conference on computer vision, pages 2551–2559, 2015

    2015

  25. [33]

    Domain generalization using causal matching

    Divyat Mahajan, Shruti Tople, and Amit Sharma. Domain generalization using causal matching. In International conference on machine learning, pages 7313–7324. PMLR, 2021

    2021

  26. [34]

    Domain generalization via invariant feature representation

    Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In International conference on machine learning, pages 10–18. PMLR, 2013

    2013

  27. [35]

    Fishr: Invariant gradient variances for out-of- distribution generalization

    Alexandre Rame, Corentin Dancette, and Matthieu Cord. Fishr: Invariant gradient variances for out-of- distribution generalization. In International Conference on Machine Learning, pages 18347–18377. PMLR, 2022

    2022

  28. [36]

    Invariant models for causal transfer learning

    Mateo Rojas-Carulla, Bernhard Schölkopf, Richard Turner, and Jonas Peters. Invariant models for causal transfer learning. Journal of Machine Learning Research, 19(36):1–34, 2018. 12

    2018

  29. [37]

    Gradient matching for domain generalization

    Yuge Shi, Jeffrey Seely, Philip HS Torr, N Siddharth, Awni Hannun, Nicolas Usunier, and Gabriel Synnaeve. Gradient matching for domain generalization. arXiv preprint arXiv:2104.09937, 2021

    2021 arXiv

  30. [38]

    Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.arXiv preprint arXiv:1911.08731, 2019

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.arXiv preprint arXiv:1911.08731, 2019

    1911 arXiv

  31. [39]

    Learning to generate novel domains for domain generalization

    Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Learning to generate novel domains for domain generalization. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16, pages 561–578. Springer, 2020

    2020

  32. [40]

    Explore and exploit the diverse knowledge in model zoo for domain generalization

    Yimeng Chen, Tianyang Hu, Fengwei Zhou, Zhenguo Li, and Zhi-Ming Ma. Explore and exploit the diverse knowledge in model zoo for domain generalization. In International Conference on Machine Learning, pages 4623–4640. PMLR, 2023

    2023

  33. [41]

    Model ratatouille: Recycling diverse models for out-of-distribution generalization

    Alexandre Ramé, Kartik Ahuja, Jianyu Zhang, Matthieu Cord, Léon Bottou, and David Lopez-Paz. Model ratatouille: Recycling diverse models for out-of-distribution generalization. In International Conference on Machine Learning, pages 28656–28679. PMLR, 2023

    2023

  34. [42]

    Test-time style shifting: Handling arbitrary styles in domain generalization

    Jungwuk Park, Dong-Jun Han, Soyeong Kim, and Jaekyun Moon. Test-time style shifting: Handling arbitrary styles in domain generalization. In International Conference on Machine Learning , pages 27114–27131. PMLR, 2023

    2023

  35. [43]

    Improved test-time adaptation for domain generalization

    Liang Chen, Yong Zhang, Yibing Song, Ying Shan, and Lingqiao Liu. Improved test-time adaptation for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24172–24182, 2023

    2023

  36. [44]

    Reducing domain gap by reducing style bias

    Hyeonseob Nam, HyunJae Lee, Jongchan Park, Wonjun Yoon, and Donggeun Yoo. Reducing domain gap by reducing style bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8690–8699, 2021

    2021

  37. [45]

    Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection

    Liang Chen, Yong Zhang, Yibing Song, Lingqiao Liu, and Jue Wang. Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18710–18719, 2022

    2022

  38. [46]

    Selfreg: Self-supervised contrastive regularization for domain generalization

    Daehee Kim, Youngjun Yoo, Seunghyun Park, Jinkyu Kim, and Jaekoo Lee. Selfreg: Self-supervised contrastive regularization for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9619–9628, 2021

    2021

  39. [47]

    A simple feature augmentation for domain generalization

    Pan Li, Da Li, Wei Li, Shaogang Gong, Yanwei Fu, and Timothy M Hospedales. A simple feature augmentation for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8886–8895, 2021

    2021

  40. [48]

    Domain generalization with mixstyle

    Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. arXiv preprint arXiv:2104.02008, 2021

    2021 arXiv

  41. [49]

    A fourier-based framework for domain generalization

    Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. A fourier-based framework for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14383–14392, 2021

    2021

  42. [50]

    Improve unsupervised domain adaptation with mixup training

    Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, and Liu Ren. Improve unsupervised domain adaptation with mixup training. arXiv preprint arXiv:2001.00677, 2020

    2001 arXiv

  43. [51]

    Metanorm: Learning to normalize few-shot batches across domains

    Yingjun Du, Xiantong Zhen, Ling Shao, and Cees GM Snoek. Metanorm: Learning to normalize few-shot batches across domains. In International Conference on Learning Representations, 2020

    2020

  44. [52]

    Open domain generalization with domain-augmented meta-learning

    Yang Shu, Zhangjie Cao, Chenyu Wang, Jianmin Wang, and Mingsheng Long. Open domain generalization with domain-augmented meta-learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9624–9633, 2021

    2021

  45. [53]

    An investigation of why overparam- eterization exacerbates spurious correlations

    Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparam- eterization exacerbates spurious correlations. In International Conference on Machine Learning, pages 8346–8356. PMLR, 2020

    2020

  46. [54]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635, 2018

    2018 arXiv

  47. [55]

    Can subnetwork structure be the key to out-of-distribution generalization? In International Conference on Machine Learning, pages 12356–12367

    Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang, and Aaron Courville. Can subnetwork structure be the key to out-of-distribution generalization? In International Conference on Machine Learning, pages 12356–12367. PMLR, 2021. 13

    2021

  48. [56]

    Training debiased subnetworks with contrastive weight pruning

    Geon Yeong Park, Sangmin Lee, Sang Wan Lee, and Jong Chul Ye. Training debiased subnetworks with contrastive weight pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7929–7938, 2023

    2023

  49. [57]

    Nas- ood: Neural architecture search for out-of-distribution generalization

    Haoyue Bai, Fengwei Zhou, Lanqing Hong, Nanyang Ye, S-H Gary Chan, and Zhenguo Li. Nas- ood: Neural architecture search for out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8320–8329, 2021

    2021

  50. [58]

    Alphanet: Improved training of supernets with alpha-divergence

    Dilin Wang, Chengyue Gong, Meng Li, Qiang Liu, and Vikas Chandra. Alphanet: Improved training of supernets with alpha-divergence. In International Conference on Machine Learning, pages 10760–10771. PMLR, 2021

    2021

  51. [59]

    Attentivenas: Improving neural architecture search via attentive sampling

    Dilin Wang, Meng Li, Chengyue Gong, and Vikas Chandra. Attentivenas: Improving neural architecture search via attentive sampling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6418–6427, 2021

    2021

  52. [60]

    Meco: Zero-shot nas with one data and single forward pass via minimum eigenvalue of correlation

    Tangyu Jiang, Haodi Wang, and Rongfang Bie. Meco: Zero-shot nas with one data and single forward pass via minimum eigenvalue of correlation. Advances in Neural Information Processing Systems, 36, 2024

    2024

  53. [61]

    Neural architecture design and robustness: A dataset

    Steffen Jung, Jovita Lukasik, and Margret Keuper. Neural architecture design and robustness: A dataset. arXiv preprint arXiv:2306.06712, 2023

    2023 arXiv

  54. [62]

    Generalizable lightweight proxy for robust nas against diverse perturbations

    Hyeonjeong Ha, Minseon Kim, and Sung Ju Hwang. Generalizable lightweight proxy for robust nas against diverse perturbations. Advances in Neural Information Processing Systems, 36, 2024

    2024

  55. [63]

    Robust nas under adversarial training: benchmark, theory, and beyond

    Yongtao Wu, Fanghui Liu, Carl-Johann Simon-Gabriel, Grigorios G Chrysos, and V olkan Cevher. Robust nas under adversarial training: benchmark, theory, and beyond. arXiv preprint arXiv:2403.13134, 2024

    2024 arXiv

  56. [64]

    Glit: Neural architecture search for global and local image transformer

    Boyu Chen, Peixia Li, Chuming Li, Baopu Li, Lei Bai, Chen Lin, Ming Sun, Junjie Yan, and Wanli Ouyang. Glit: Neural architecture search for global and local image transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–21, 2021

    2021

  57. [65]

    Shiftnas: Improving one-shot nas via probability shift

    Mingyang Zhang, Xinyi Yu, Haodong Zhao, and Linlin Ou. Shiftnas: Improving one-shot nas via probability shift. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5919–5928, 2023

    2023

  58. [66]

    Mdl-nas: A joint multi-domain learning framework for vision transformer

    Shiguang Wang, Tao Xie, Jian Cheng, Xingcheng Zhang, and Haijun Liu. Mdl-nas: A joint multi-domain learning framework for vision transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20094–20104, 2023

    2023

  59. [67]

    Efficient multimodal fusion via interactive prompting

    Yaowei Li, Ruijie Quan, Linchao Zhu, and Yi Yang. Efficient multimodal fusion via interactive prompting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2604–2613, 2023

    2023

  60. [68]

    Imagenet: Constructing a large-scale image database

    Li Fei-Fei, Jia Deng, and Kai Li. Imagenet: Constructing a large-scale image database. Journal of vision, 9(8):1037–1037, 2009

    2009

  61. [69]

    Accuracy on the line: on the strong correlation between out-of- distribution and in-distribution generalization

    John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy on the line: on the strong correlation between out-of- distribution and in-distribution generalization. In Marina Meila and Tong...

    2021

  62. [70]

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do ImageNet classifiers generalize to ImageNet? In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Ma...

    2019

  63. [71]

    Id and ood performance are sometimes inversely correlated on real-world datasets

    Damien Teney, Yong Lin, Seong Joon Oh, and Ehsan Abbasnejad. Id and ood performance are sometimes inversely correlated on real-world datasets. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, vo...

    2023

  64. [72]

    Assaying Out-Of-Distribution Generalization in Transfer Learning

    Florian Wenzel, Andrea Dittadi, Peter Gehler, Carl-Johann Simon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, Chris Russell, Thomas Brox, Bernt Schiele, Bernhard Schölkopf, and Francesco Locatello. Assaying Out-Of-Distribution Generalization in Transfer Learning. In S Koy...

    2022

  65. [73]

    Snip: Single-shot network pruning based on connection sensitivity

    Namhoon Lee, Thalaiyasingam Ajanthan, and Philip HS Torr. Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340, 2018

    2018 arXiv

  66. [74]

    Picking winning tickets before training by preserving gradient flow

    Chaoqi Wang, Guodong Zhang, and Roger Grosse. Picking winning tickets before training by preserving gradient flow. arXiv preprint arXiv:2002.07376, 2020

    2002 arXiv

  67. [75]

    Dsrna: Differentiable search of robust neural architectures

    Ramtin Hosseini, Xingyi Yang, and Pengtao Xie. Dsrna: Differentiable search of robust neural architectures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6196– 6205, 2021

    2021

  68. [76]

    A new measure of rank correlation

    Maurice G Kendall. A new measure of rank correlation. Biometrika, 30(1/2):81–93, 1938

    1938

  69. [77]

    Improving vision transform- ers by revisiting high-frequency components

    Jiawang Bai, Li Yuan, Shu-Tao Xia, Shuicheng Yan, Zhifeng Li, and Wei Liu. Improving vision transform- ers by revisiting high-frequency components. In European Conference on Computer Vision, pages 1–18. Springer, 2022

    2022

  70. [78]

    On the adversarial robustness of vision transformers

    Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin-Yu Chen, and Cho-Jui Hsieh. On the adversarial robustness of vision transformers. arXiv preprint arXiv:2103.15670, 2021

    2021 arXiv

  71. [79]

    Can biases in imagenet models explain generalization? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22184–22194, 2024

    Paul Gavrikov and Janis Keuper. Can biases in imagenet models explain generalization? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22184–22194, 2024

    2024

  72. [80]

    High-frequency component helps explain the generalization of convolutional neural networks

    Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P Xing. High-frequency component helps explain the generalization of convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8684–8694, 2020

    2020

  73. [81]

    Arbitrary style transfer in real-time with adaptive instance normalization

    Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE international conference on computer vision, pages 1501–1510, 2017

    2017

  74. [82]

    cat", "airplane

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 15 6 Appendix Overview...

    2022

  75. [83]

    These examples could be distorted or unrealistic in object- background placements

    and only hard examples are kept. These examples could be distorted or unrealistic in object- background placements. We visualize a few examples of OoD datasets in Fig. 10.1. For ImageNet-C, we provide the visualization of different OoD shift severity in Fig. 10.3. 19 Figure 10...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.