Pith. sign in

REVIEW 3 major objections 6 minor 182 references

Smaller, Faster, Cheaper: Architectural Designs for Efficient Machine Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Architectural design, not raw scale, can make neural networks smaller, faster, and cheaper without sacrificing performance, the dissertation argues, and it offers three concrete interventions as evidence.

desk verdict A well-organized thesis compilation of three solid efficiency papers, but the Flowers-102 split gap undercuts the headline data-efficiency claim. read the letter →

arxiv 2507.19795 v1 pith:M7EPWACO submitted 2025-07-26 cs.CV cs.ARcs.LG

Steven Walton This is my paper
classification cs.CVcs.ARcs.LG
keywords visiontransformerscompactneighborhoodattentionvariadicheadsimagegenerationnormalizingflowsknowledgedistillationmodelefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The dissertation tries to establish that careful architectural design can substitute for scale: models can be made smaller, faster, and cheaper without losing performance. It defends this with three results: compact convolutional transformers trained from scratch match or beat much larger pretrained vision transformers on small and medium image benchmarks; giving each attention head its own window size and dilation in neighborhood attention yields a StyleGAN-style generator that reaches FID 2.05 on FFHQ-256 with 48.9M parameters; and a taxonomy of normalizing-flow distillation transfers most of a teacher's density-estimation ability to a student a fraction of its size. If correct, efficiency becomes a design property rather than a brute-force scaling outcome, which would matter for deploying models on limited hardware and small datasets.

What carries the argument

The argument is carried by three named mechanisms. First, the Compact Convolutional Transformer tokenizer: an overlapping convolutional patch embedding, with kernel size larger than stride followed by max pooling, that preserves boundary information, paired with SeqPool, a learned attention-inspired pooling that contracts the token sequence into a vector instead of slicing a single class token. Second, Hydra Neighborhood Attention: a restricted-attention mechanism in which each attention head has its own window size and dilation, so dense local and sparse global receptive fields intermix; it reduces to standard neighborhood attention when heads share one kernel and to full self-attention when the window covers the image. Third, compositional invertibility of normalizing flows: because a flow is a composition of diffeomorphic layers, knowledge can be transferred as latent distillation, intermediate-latent distillation, and synthesized distillation, with a combined loss weighting each channel.

What would settle it

Retrain CCT-14/7×2 on the standard Flowers-102 split using the dissertation's exact recipe; if it yields roughly 68.85% instead of 97.19%, the Flowers-102 evidence for escaping the big-data paradigm fails on that split. Also rerun the CIFAR-10 CCT-7/3×1 experiments with a validation split fixed before any hyperparameter search; if the reported 98% drops materially, part of the gain is selection on the test set.

Watch

Extended reading notes

Core claim

On the dissertation's own terms, the core claim is that three architectural interventions each improve accuracy or likelihood per unit of compute and data in their settings, and together they answer affirmatively whether neural architectures can be smaller, faster, and cheaper without sacrificing performance. Chapter 3 shows that a Compact Convolutional Transformer with an overlapping convolutional tokenizer and SeqPool reaches 98.00% on CIFAR-10 with 3.76M parameters when trained from scratch, outperforming much larger ViTs and ResNets, and reaches 81.34% top-1 on ImageNet with distillation against DeiT-S's 81.16%. Chapter 4 shows that variadic attention heads, giving each head an independent window size and dilation, let a StyleGAN-style generator score FID 2.05 on FFHQ-256 at 48.92M parameters and 59.90 images per second, beating StyleSwin's 2.81 and StyleGAN-XL's 2.19 at lower parameter cost and higher throughput. Chapter 5 formalizes flow distillation into latent, intermediate-latent, and synthesized channels, showing a GLOW student with about a quarter of the teacher's parameters retaining roughly 99% of its BSDS300 log-likelihood, and intermediate-latent distillation improving image-generation FID on both CIFAR-10 and CelebA.

Load-bearing premise

The load-bearing premise is that the reported benchmark numbers are unbiased held-out estimates of generalization, but hyperparameters were selected by validation-set search with best results reported, and the Flowers-102 97.19% headline came from the Kaggle split while the standard split yields 68.85%, so if the premise fails the small-data claim loses its headline evidence.

Editorial extensions

If this is right

  • Vision transformers can be trained from scratch on small and medium image datasets and still beat comparable ViTs and ResNets, so web-scale pretraining is not a prerequisite for transformer adoption in vision.
  • Allowing attention heads to use independent window sizes and dilations lets generative models mix local and global structure, which improves FID at comparable parameter count and throughput on FFHQ-256.
  • Because normalizing flows are compositions of invertible layers, teacher knowledge can be transferred in three directions, so a flow student a fraction of the teacher's size can retain most of its density-estimation performance.
  • Efficiency gains can come from data ingress and egress, core attention design, and structure-aware distillation, so scaling data and parameters is not the only route to progress.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the Hydra-NA idea should transfer to other generative backbones, especially diffusion models where local texture and global layout both matter; swapping a latent diffusion model's attention blocks for variadic heads would test this directly.
  • Editorial inference: the Flowers-102 split sensitivity suggests small-dataset transformer results may be fragile to dataset construction, so standardizing evaluation splits would make the escape-from-big-data claim falsifiable across labs.
  • Editorial inference: the three flow-distillation channels could in principle extend to continuous flow-matching models by aligning trajectories in time rather than composing layers, though the dissertation only gestures at this possibility.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This dissertation argues that careful architectural design can make vision and generative models smaller, faster, and cheaper without sacrificing performance, and it presents three supporting interventions. Chapter 3 introduces Compact Convolutional Transformers (CCTs), which combine an overlapping convolutional tokenizer with SeqPool, and reports strong results on small datasets, ImageNet, and Flowers-102. Chapter 4 extends Neighborhood Attention to allow attention heads to have independent kernel sizes and dilations (Hydra-NA), and demonstrates improved FID, throughput, and parameter counts for StyleGAN-style image generation on FFHQ and LSUN Church. Chapter 5 formalizes three categories of knowledge distillation for normalizing flows (latent, intermediate-latent, and synthesized) and shows improved density estimation and image generation over a non-distilled student. The dissertation concludes that these results positively answer the question of whether architectures can be designed to be smaller, faster, and cheaper without sacrificing performance.

Significance. If the reported results hold, the thesis makes a useful empirical case that targeted architectural modifications (overlapping tokenization, per-head receptive fields, and flow-structure-aware distillation) can shift the efficiency-accuracy Pareto frontier. The work has several strengths: the CCT and StyleNAT code and checkpoints are released; the author is transparent about the Flowers-102 split problem and about the non-convergence of some StyleNAT runs; and the attention-map analysis and metric critique in Chapter 4 are thoughtful. The main significance is dampened by the evaluation-protocol issues in Chapter 3, which is the chapter most directly tied to the dissertation's 'escape the big-data paradigm' claim.

major comments (3)
  1. [Section 3.3.6, Table 8] The headline Flowers-102 result of 97.19% for CCT-14/7x2 without pretraining is obtained on the Kaggle split, and the text immediately discloses that retraining on the standard torchvision split gives 68.85%. A 28-point swing due to the split choice is far larger than the efficiency margins the chapter claims, so the statements 'able to achieve an accuracy of over 97% without the use of any pretraining data' and 'state of the art results' (99.76% with pretraining) are not supported on the standard benchmark. The chapter should report the standard-split numbers as primary and either remove the Kaggle-split claims or clearly re-label them as non-standard; the Section 6.1 summary should be revised to match.
  2. [Section 3.3.3, Section 3.3.5, Table 6] The text states that hyperparameters were selected by directed search on a validation set and that 'the best results we achieved' are reported, while Table 6 notes that numbers are 'best out of 4 runs' with random initializations. Because no separate validation split and no measure of run-to-run variability are provided, the reported accuracies in Table 3 are selected, not typical, estimates. This matters for Chapter 3's data-efficiency claims, several of which rest on small margins (e.g., CCT-7/3x1 vs. ResNet164 on CIFAR-100). Please report mean and standard deviation over the repeated runs, or at minimum the full set of per-run numbers, and make explicit which split was used for model selection.
  3. [Section 5.3.1, Section 5.3.2, Table 17] The claim that the GLOW student 'achieves 98.94% the accuracy' of the teacher on BSDS300 is based only on test log-likelihood. In the image-generation experiments the same method leaves a much larger relative gap: the CelebA teacher FID is 37.460, the non-distilled student is 68.127, and the ILKD student is 54.480. The text should temper the 'passing nearly all its knowledge' conclusion and present the interpolation-based FID of Table 18 as evidence of the transfer, since the raw FID gap indicates significant remaining quality loss.
minor comments (6)
  1. [Section 3.3.6 and Table 8 caption] The first paragraph of Section 3.3.6 states 'we are able to achieve an accuracy of over 97%' without the caveat about the Kaggle split, which only appears later in the section and in the table note; the caveat should accompany the first mention.
  2. [Section 3.3.5, Figure 9] The subcaption text for Figures 9b and 9c appears swapped: the text says 'Fig. 9c with sinusoidal' and 'Fig. 9b with a learnable positional embedding,' but the panels are captioned the reverse. Please check and correct the cross-references.
  3. [Chapter 2 text] The document contains several typographical errors, including 'attnetion' (Section 2.3.1), 'explinations' (Acknowledgements), 'Labratory' (Curriculum Vitae), and 'CIF AR-10' spacing throughout; a copyedit pass is needed.
  4. [Section 5.3.1] The sentence 'our final student GLOW model is has ≈25.5 as many parameters as the teacher model' is missing the percentage sign: it should read '≈25.5% as many parameters.'
  5. [Section 5.2.1.4, Eq. (5.9)] The loss weights λ_i are introduced without a normalization convention; the experimental section says they are 'percentages of the whole loss,' but this is not reflected in Equation (5.9), so please clarify whether the weights sum to one or are scaled elsewhere.
  6. [Chapter 3, Table 6] The main-results Table 3 does not state whether those numbers are also 'best out of 4 runs' like Table 6; please add an explicit statement about run selection for all tables in Chapter 3.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the dissertation's claims are empirical and benchmark-driven; disclosed evaluation-protocol caveats are validity concerns, not derivation-circularity.

full rationale

This dissertation does not present a formal derivation chain whose conclusions reduce to its inputs; the three contributions are architectural modifications evaluated against external baselines. Chapter 3's CCT is defined by explicit tokenizer and pooling equations (3.1)-(3.2) and compared to ResNet/ViT/DeiT baselines; Chapter 4's variadic attention heads are defined by (4.3)-(4.4) and ablated against StyleSwin and StyleGAN variants; Chapter 5's flow-distillation framework is defined by (5.6)-(5.9) and tested against teacher/student GLOW and MAF models. The main evaluation-protocol weaknesses are explicitly acknowledged in the text: hyperparameters were selected by directed search and the best of 4 runs are reported (Sections 3.3.3 and Table 6 caption, and Section 6.2.3.1), and the Flowers-102 result depends on the Kaggle split, with the torchvision-split retraining giving 68.85% rather than 97.19% (Section 3.3.6). These are real threats to the trustworthiness of the reported generalization estimates, but they are selection-bias and benchmark-validity problems, not circularity: no reported number is equivalent by construction to a fitted input, and no load-bearing claim is justified solely by an unverified self-citation. Self-citations to the authors' prior NAT/CCT works serve as background or as the same works being reported, with code and experimental details included, so they do not make the argument circular.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper modifies architectures and training objectives but introduces no new particles, forces, dimensions, or other postulated entities. The Hydra-NA mechanism is a configuration of existing attention, not a new entity; therefore the invented-entities ledger is empty.

free parameters (4)
  • CCT tokenizer hyperparameters (kernel, stride, number of convolutions) = 3x3 kernel, stride 1, 1 convolution for CCT-7/3x1; variants use 7x7 or 3x3 with strides 1 or 2
    Selected by directed search optimizing validation accuracy (Sections 3.3.3 and 6.2.3.1); reported CIFAR numbers are the best found under this search.
  • StyleNAT attention head kernel and dilation schedule = kernel 7 with dilations 1,2,4,...,128 depending on resolution (Table 10); 8 splits for LSUN Church
    Hand-selected from a subset of the 13,176 possible configurations on FFHQ-256; the chosen schedule became the headline result, so part of the gain reflects configuration choice.
  • Distillation loss weights lambda_i = lambda0=0.9, lambda1=lambda2=0.1; for SKD lambda0=0.85 and the others 0.075
    Chosen by hand and described as likely non-optimal in Section 5.3.1; they directly control the magnitude of the reported distillation gains.
  • StyleNAT training iterations and LR-decay start = LR-decay at 740k, stopped at 940k for FFHQ-256; 500k and 900k for FFHQ-1024
    Selected based on observed FID curves (Section 4.5, Figure 16); the paper notes the model had not converged, so these stopping points influence the reported FID values.
assumptions (5)
  • domain assumption Reported CIFAR-10/100 and Flowers-102 accuracies are unbiased held-out estimates, with no separate validation split used for model selection.
    The text reports hyperparameter sweeps and best results; for Flowers-102 the headline table uses a Kaggle split and standard-split retraining gives 68.85% (Section 3.3.6). If the test set was used for selection, the numbers are optimistic.
  • domain assumption The student's latent representation is at least as large as the latent data manifold, making full distillation possible.
    Stated explicitly in Section 5.3.1 as the condition under which teacher knowledge can be fully transferred; the paper does not verify the condition.
  • domain assumption The latent manifold hypothesis: information needed to generate an image is smaller than the image's dimensionality.
    Used to justify choosing StyleGAN's progressive structure as the experimental platform in Section 4.4; standard but unproved.
  • domain assumption Neighborhood attention with per-head dilation can emulate global attention when the dilated window covers the image.
    Inherited from the cited NAT and DiNA works [52,50] and used to justify that variadic heads recover global context; accepted from prior literature rather than re-derived here.
  • standard math Change-of-variables formula for bijective differentiable maps
    Used in Section 2.3.3 and Chapter 5 to define normalizing flow likelihoods; standard calculus background.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Smaller, Faster, Cheaper: Architectural Designs for Efficient Machine Learning." pith.science (2026). https://pith.science/paper/M7EPWACO

@misc{pith2026250719795,
  author       = {Pith},
  title        = {Pith review of: Smaller, Faster, Cheaper: Architectural Designs for Efficient Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M7EPWACO}},
  note         = {Machine review of arXiv:2507.19795}
}
read the original abstract

Major advancements in the capabilities of computer vision models have been primarily fueled by rapid expansion of datasets, model parameters, and computational budgets, leading to ever-increasing demands on computational infrastructure. However, as these models are deployed in increasingly diverse and resource-constrained environments, there is a pressing need for architectures that can deliver high performance while requiring fewer computational resources. This dissertation focuses on architectural principles through which models can achieve increased performance while reducing their computational demands. We discuss strides towards this goal through three directions. First, we focus on data ingress and egress, investigating how information may be passed into and retrieved from our core neural processing units. This ensures that our models make the most of available data, allowing smaller architectures to become more performant. Second, we investigate modifications to the core neural architecture, applied to restricted attention in vision transformers. This section explores how removing uniform context windows in restricted attention increases the expressivity of the underlying neural architecture. Third, we explore the natural structures of Normalizing Flows and how we can leverage these properties to better distill model knowledge. These contributions demonstrate that careful design of neural architectures can increase the efficiency of machine learning algorithms, allowing them to become smaller, faster, and cheaper.

Figures

Figures reproduced from arXiv: 2507.19795 by the authors.

Figure 1
Figure 1. The diagram illustrating concepts from Set Theory, explaining the Domain (D), Codomain, Range (R), Image (Re), and Preimage. is important to note that the Preimage is not the inverse of the image. Many texts use the notation f −1 , but we will use f ∗ to avoid confusion.1 [PITH_FULL_IMAGE:figures/full_fig_p026_1.png] view at source ↗
Figure 2
Figure 2. The Transformer model architecture from Vaswani et. al. Diagram depicts dot-product self attention. In [PITH_FULL_IMAGE:figures/full_fig_p032_2.png] view at source ↗
Figure 3
Figure 3. Taxonomy of Generative Models, based on Goodfellow’s Taxonomy [40] this equation is a softmax temperature scale, which is the inverse square root of the embedding dimension (a user defined hyperparameter). The queries and keys are multiplied together, learning a similarity matrix. The softmax of this is then referred to as the “score”, as its values are defined by a probability distribution. The value tensor is then… view at source ↗
Figures from the paper (22 more)
Figure 4
Figure 4. Figure 4: Visual representation of injections, surjections , and bijections. Source: Wolfram Mathworld The two most common forms of Normalizing Flows, which are also used within this thesis, are: Affine coupling flows. : Partition input x into two units, (x0, x1), such that f(x0…
Figure 5
Figure 5. Figure 5: Architectural design of Compact Transformers Bert [24] style Transformer architecture could learn convolutional filters, given a sufficient number of attention heads. Unfortunately, these researchers were memory bound and were using 2 × 2 invertible down-sampling. Doso…
Figure 6
Figure 6. Figure 6: A comparison of the Vision Transformer variants used throughout this study. On the left is the batching and embedding process (tokenization). On the right is the main neural architecture. The Transformer Encoder blocks and Linear Layers (classification sub-network) are…
Figure 7
Figure 7. Figure 7: Salient maps of ViT, CCT, and NAT based on ImageNet-1k. It can be seen that CCT removes the blocking artifacts from ViT. CCT sometimes creates displacement, but this is resolved by NAT (presented in Chapter 4). 52 [PITH_FULL_IMAGE:figures/full_fig_p052_7.png]
Figure 8
Figure 8. Figure 8: Comparison of models with restricted number of samples per class. At 10% models are trained on only 5000 images. Transformer based models demonstrate better scalability than ResNet based models. Moving on to further test the scalability of our model with respect to dat…
Figure 9
Figure 9. Figure 9: Comparison of ViT-Lite and CCT accuracy on CIFAR-10 with differing image resolutions. In first row, models are trained from scratch. In second row, models are inference and trained on 32 × 32 images. Fig. 9a is without positional embedding, Fig. 9c with sinusoidal posi…
Figure 10
Figure 10. Figure 10: Samples form FFHQ-256 (left) with FID: 2.05, FFHQ-1024 (center) with FID: 4.17, and Church (right) with FID: 3.40 generated by our StyleNAT network, using Hydra Neighborhood Attention. 66 [PITH_FULL_IMAGE:figures/full_fig_p066_10.png]
Figure 11
Figure 11. Figure 11: Comparison of Neighborhood Attention, Swin, and ConvNeXt on ImageNet classification. To resolve these issues, Hassani et al. developed the Neighborhood Attention Transformer (NAT) [52]. The architecture is similar to SASA but resolved the generalization issue, ensurin…
Figure 12
Figure 12. Figure 12: Diagram depicting the Neighborhood Attention Transformer [52] as applied to vision classification. 4.3 Variadic Attention Heads A subtle feature of multi-headed attention [150] is that attention heads are independent of one another. This property allows each head to a…
Figure 13
Figure 13. Figure 13: StyleNAT Architecture To evidence our hypothesis presents a challenge, due to the nature of most neural architectures accounting for these limitations and mixing data as dept increases. With limited compute infrastructure there is significant pressure to design the ri…
Figure 14
Figure 14. Figure 14: StyleNAT represents the Pareto Frontier for FID (y-axis), Parameters (bubble size), and throughput (x-axis) on FFHQ-256. StyleNAT has the lowest FID of 2.05, with fewer parameters (48.92M) than similarly performing models and is capable of generating images in real-ti…
Figure 15
Figure 15. Figure 15: Samples generated by StyleNAT. We do not use truncation, softmax tempering, nor any other such enhancement techniques. but set half the heads to have a dilation increasing by a power of 2 (d = 2N ), maximally for the resolution at a given level, where N =  log2 R k …
Figure 16
Figure 16. Figure 16: FFHQ training: FID vs Iteration (in thousands). We see that the FID performance has not converged. This suggests the models are not optimally trained. but was not used on FFHQ and decided to perform this training as NA had shown to be quasi-equivariant to translations…
Figure 17
Figure 17. Figure 17: Visual artifacts from StyleGAN3 FFHQ-1024 samples (using image 0068). Sample highlights banding effects, hexagonal patterns, and other artifacts common to this generator. Within the StyleGAN images we notice a string of beads like artifacts. These structures may be di…
Figure 18
Figure 18. Figure 18: Visual artifacts from StyleSwin FFHQ-1024 samples (we generated these). Sample highlights rectangular geometric patterns on face, and poor texture on ears. We found that StyleSwin’s integration of sliding windows (SWA) and shifted windows (SWSA) does not properly inte…
Figure 19
Figure 19. Figure 19: Visual artifacts from StyleNAT FFHQ-1024 samples (We generated these). Sample highlights minor skin texture issues, some chromatic aberrations, and unnatural blue speckling around eyes. While this sample has many artifacts and may be more easily identified than the St…
Figure 20
Figure 20. Figure 20: Visualization of the first and last attention head progressing through StyleNAT. We start at a resolution of 16×16 and grow to 1024×1024. We generate 50 samples from each network and choose the best image from the sample to make comparisons as fair as possible. The to…
Figure 21
Figure 21. Figure 21: Visualization of Attention maps (Figs. 21a and 21b StyleNAT, Figs. 21c and 21d StyleSwin) for transformers at the 1024 resolution. Top row corresponds to localized dense kernels (k = 7, d = 1), second row corresponds to the sparse dilated kernels (k = 7, d = 128). Ban…
Figure 22
Figure 22. Figure 22: Illustration of knowledge transfer between two Glow [86] based models. LLKD represents the Latent Knowledge transfer between the learned representations. LILKD is the knowledge transfer between intermediate representations. LSKD is the knowledge transfer via synthesiz…
Figure 22
Figure 22. Figure 22: For MAF we match across depths, pushing every two depths from the [PITH_FULL_IMAGE:figures/full_fig_p105_22.png]
Figure 23
Figure 23. Figure 23: CIFAR-10 samples from teacher model (Fig. 23a), student model (Fig. 23b), and student model with no knowledge distillation (Fig. 23c). All images are generated at 32 × 32 resolution and with a temperature of 0.7. Considering the biases of FID, high quality samples can…
Figure 24
Figure 24. Figure 24: CelebA samples from teacher model (Fig. 24a), student model (Fig. 24b), and student model with no knowledge distillation (Fig. 24c). All images are generated at 64×64 resolution and with temperature=0.7. 5.4 Conclusion Our work we sought to build foundations for inves…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

182 extracted references · 55 canonical work pages

  1. [1]

    Semdedup: Data-efficient learning at web-scale through semantic deduplication

    Amro Abbas, Kushal Tirumala, D´ aniel Simig, Surya Ganguli, and Ari S Morcos. Semdedup: Data-efficient learning at web-scale through semantic deduplication. arXiv preprint arXiv:2303.09540 , 2023

    arXiv 2023

  2. [2]

    Dbpedia: A nucleus for a web of open data

    S¨ oren Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. Dbpedia: A nucleus for a web of open data. In The semantic web, pages 722–735. Springer, 2007

    2007

  3. [3]

    Neural machine translation by jointly learning to align and translate, 2016

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate, 2016

    2016

  4. [4]

    Distilling the knowledge from conditional normalizing flows

    Dmitry Baranchuk, Vladimir Aliev, and Artem Babenko. Distilling the knowledge from conditional normalizing flows. In ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models , 2021

    2021

  5. [5]

    Semantic photo manipulation with a generative image prior

    David Bau, Hendrik Strobelt, William Peebles, Jonas Wulff, Bolei Zhou, Jun-Yan Zhu, and Antonio Torralba. Semantic photo manipulation with a generative image prior. ACM Transactions on Graphics , 38(4):1–11, 2019

    2019

  6. [6]

    Irwan Bello, Barret Zoph, Ashish Vaswani, Jonathon Shlens, and Quoc V. Le. Attention augmented convolutional networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , 2019

    2019

  7. [7]

    Longformer: The long-document transformer

    Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150 , 2020

    arXiv 2004

  8. [8]

    Tomczak, and Max Welling

    Rianne van den Berg, Leonard Hasenclever, Jakub M. Tomczak, and Max Welling. Sylvester Normalizing Flows for Variational Inference, 2019. arXiv:1803.05649 [cs, stat]. 124

    arXiv 2019

Show all 182 references
  1. [9]

    Unleashing transformers: Parallel token prediction with discrete absorbing diffusion for fast high-resolution image generation from vector-quantized codes

    Sam Bond-Taylor, Peter Hessey, Hiroshi Sasaki, Toby P Breckon, and Chris G Willcocks. Unleashing transformers: Parallel token prediction with discrete absorbing diffusion for fast high-resolution image generation from vector-quantized codes. In European Conference on Computer ...

    2022

  2. [10]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffr...

    2020

  3. [11]

    Model compression

    Cristian Bucilu˘a, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 535–541, 2006

    2006

  4. [12]

    Proxylessnas: Direct neural architecture search on target task and hardware

    Han Cai, Ligeng Zhu, and Song Han. Proxylessnas: Direct neural architecture search on target task and hardware. In ICLR, 2018

    2018

  5. [13]

    Cascade r-cnn: Delving into high quality object detection

    Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection. In CVPR, 2018

    2018

  6. [14]

    Ricky T. Q. Chen and Yaron Lipman. Flow matching on general geometries, 2024

    2024

  7. [15]

    Ricky T. Q. Chen, Jens Behrmann, David Duvenaud, and J¨ orn-Henrik Jacobsen. Residual Flows for Invertible Generative Modeling, 2020. arXiv:1906.02735 [cs, stat]. 125

    2020 arXiv

  8. [16]

    On the relationship between self-attention and convolutional layers

    Jean-Baptiste Cordonnier, Andreas Loukas, and Martin Jaggi. On the relationship between self-attention and convolutional layers. In International Conference on Learning Representations, 2020

    2020

  9. [17]

    Scalable high-resolution pixel-space image synthesis with hourglass diffusion transformers

    Katherine Crowson, Stefan Andreas Baumann, Alex Birch, Tanishq Mathew Abraham, Daniel Z Kaplan, and Enrico Shippole. Scalable high-resolution pixel-space image synthesis with hourglass diffusion transformers. In Forty-first International Conference on Machine Learning , 2024

    2024

  10. [18]

    Approximation with artificial neural networks.Faculty of Sciences, Etvs Lornd University, Hungary , 24(48):7, 2001

    Bal´ azs Csan´ ad Cs´ aji et al. Approximation with artificial neural networks.Faculty of Sciences, Etvs Lornd University, Hungary , 24(48):7, 2001

    2001

  11. [19]

    Randaugment: Practical automated data augmentation with a reduced search space

    Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 702–703, 2020

    2020

  12. [20]

    Flow matching in latent space

    Quan Dao, Hao Phung, Binh Nguyen, and Anh Tran. Flow matching in latent space. arXiv preprint arXiv:2307.08698 , 2023

    2023 arXiv

  13. [21]

    Flashattention-2: Faster attention with better parallelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. In ICLR, 2023

    2023

  14. [22]

    Flashattention: Fast and memory-efficient exact attention with io-awareness

    Tri Dao, Daniel Y Fu, Stefano Ermon, Atri Rudra, and Christopher R´ e. Flashattention: Fast and memory-efficient exact attention with io-awareness. In NeurIPS, 2022

    2022

  15. [23]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009

    2009

  16. [24]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In 126 Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human l...

    2019

  17. [25]

    Ensemble methods in machine learning

    Thomas G Dietterich. Ensemble methods in machine learning. In International workshop on multiple classifier systems , pages 1–15. Springer, 2000

    2000

  18. [26]

    Density estimation using real nvp, 2017

    Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp, 2017

    2017

  19. [27]

    Dolatabadi, Sarah Erfani, and Christopher Leckie

    Hadi M. Dolatabadi, Sarah Erfani, and Christopher Leckie. Invertible Generative Modeling using Linear Rational Splines, 2020. arXiv:2001.05168 [cs, stat]

    2020 arXiv

  20. [28]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  21. [29]

    Augmented Neural ODEs

    Emilien Dupont, Arnaud Doucet, and Yee Whye Teh. Augmented Neural ODEs. In Advances in Neural Information Processing Systems . Curran Associates, Inc., 2019

    2019

  22. [30]

    Cubic- Spline Flows, 2019

    Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. Cubic- Spline Flows, 2019. arXiv:1906.02145 [cs, stat]

    2019 arXiv

  23. [31]

    Softmax linear units

    Nelson Elhage, Tristan Hume, Catherine Olsson, Neel Nanda, Tom Henighan, Scott Johnston, Sheer ElShowk, Nicholas Joseph, Nova DasSarma, Ben Mann, Danny Hernandez, Amanda Askell, Kamal Ndousse, Andy Jones, Dawn Drain, Anna Chen, Yuntao Bai, Deep Ganguli, Liane Lovitt, Zac Hatfi...

  24. [32]

    Visualizing higher-layer features of a deep network, 2009

    Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of a deep network, 2009

    2009

  25. [33]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019

    2019

  26. [34]

    Fuhr and Michael Kallay

    Richard D. Fuhr and Michael Kallay. Monotone linear rational spline interpolation. Computer Aided Geometric Design , 9(4):313–319, 1992

    1992

  27. [35]

    A new algorithm for data compression

    Philip Gage. A new algorithm for data compression. C Users J. , 12(2):23–38, 1994

    1994

  28. [36]

    Swagan: A style-based wavelet-driven generative model

    Rinon Gal, Dana Cohen Hochberg, Amit Bermano, and Daniel Cohen-Or. Swagan: A style-based wavelet-driven generative model. ACM Trans. Graph., 40(4), 2021

    2021

  29. [37]

    Masked diffusion transformer is a strong image synthesizer

    Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 23164–23173, 2023

    2023

  30. [38]

    Mahoney, and Kurt Keutzer

    Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference, 2021

    2021

  31. [39]

    Ganalyze: Toward visual definitions of cognitive image properties

    Lore Goetschalckx, Alex Andonian, Aude Oliva, and Phillip Isola. Ganalyze: Toward visual definitions of cognitive image properties. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , 2019

    2019

  32. [40]

    Deep Learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. 128

    2016

  33. [41]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems . Curran Associates, Inc., 2014

    2014

  34. [42]

    Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, and David Duvenaud. Scalable reversible generative models with free-form continuous dynamics. In International Conference on Learning Representations , 2019

    2019

  35. [43]

    Neural turing machines, 2014

    Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines, 2014

    2014

  36. [44]

    Densely connected normalizing flows, 2021

    Matej Grci´ c, Ivan Grubiˇ si´ c, and Siniˇ saˇSegvi´ c. Densely connected normalizing flows, 2021

    2021

  37. [45]

    Pattern theory: from representation to inference

    Ulf Grenander and Michael I Miller. Pattern theory: from representation to inference. OUP Oxford, 2006

    2006

  38. [46]

    Mamba: Linear-time sequence modeling with selective state spaces, 2024

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, 2024

    2024

  39. [47]

    Starflow: Scaling latent normalizing flows for high-resolution image synthesis, 2025

    Jiatao Gu, Tianrong Chen, David Berthelot, Huangjie Zheng, Yuyang Wang, Ruixiang Zhang, Laurent Dinh, Miguel Angel Bautista, Josh Susskind, and Shuangfei Zhai. Starflow: Scaling latent normalizing flows for high-resolution image synthesis, 2025

    2025

  40. [48]

    Variational Inference with Orthogonal Normalizing Flows

    Leonard Hasenclever, Jakub M Tomczak, and Max Welling. Variational Inference with Orthogonal Normalizing Flows. In Bayesian Deep Learning, 2017

    2017

  41. [49]

    Neighborhood attention: Dynamic restriction of self-attention, 2023

    Ali Hassani. Neighborhood attention: Dynamic restriction of self-attention, 2023

    2023

  42. [50]

    Dilated neighborhood attention transformer, 2023

    Ali Hassani and Humphrey Shi. Dilated neighborhood attention transformer, 2023. 129

    2023

  43. [51]

    Escaping the big data paradigm with compact transformers, 2022

    Ali Hassani, Steven Walton, Nikhil Shah, Abulikemu Abuduweili, Jiachen Li, and Humphrey Shi. Escaping the big data paradigm with compact transformers, 2022

    2022

  44. [52]

    Neighborhood attention transformer

    Ali Hassani, Steven Walton, Jiachen Li, Shen Li, and Humphrey Shi. Neighborhood attention transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 6185–6194, 2023

    2023

  45. [53]

    Faster neighborhood attention: Reducing the O(n2) cost of self attention at the threadblock level, 2024

    Ali Hassani, Wen-Mei Hwu, and Humphrey Shi. Faster neighborhood attention: Reducing the O(n2) cost of self attention at the threadblock level, 2024

    2024

  46. [54]

    Generalized neighborhood attention: Multi-dimensional sparse attention at the speed of light, 2025

    Ali Hassani, Fengzhe Zhou, Aditya Kane, Jiannan Huang, Chieh-Yun Chen, Min Shi, Steven Walton, Markus Hoehnerbach, Vijay Thakkar, Michael Isaev, Qinsheng Zhang, Bing Xu, Haicheng Wu, Wen mei Hwu, Ming-Yu Liu, and Humphrey Shi. Generalized neighborhood attention: Multi-dimensio...

    2025

  47. [55]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016

    2016

  48. [56]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016

    2016

  49. [57]

    Identity mappings in deep residual networks

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, 2016

    2016

  50. [58]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll´ ar, and Ross Girshick. Mask r-cnn. In ICCV, 2017

    2017

  51. [59]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of 130 the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 16000–16009, 2022

    2022

  52. [60]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems . Curran Associates, Inc., 2017

    2017

  53. [61]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 , 2015

    2015 arXiv

  54. [62]

    Geoffrey F. Hinton. Shape representation in parallel systems. In Proceedings of the 7th International Joint Conference on Artificial Intelligence - Volume 2 , page 1088–1096, San Francisco, CA, USA, 1981. Morgan Kaufmann Publishers Inc

    1981

  55. [63]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems , pages 6840–6851. Curran Associates, Inc., 2020

    2020

  56. [64]

    The convolution exponential and generalized sylvester flows

    Emiel Hoogeboom, Victor Garcia Satorras, Jakub Tomczak, and Max Welling. The convolution exponential and generalized sylvester flows. In Advances in Neural Information Processing Systems, pages 18249–18260. Curran Associates, Inc., 2020

    2020

  57. [65]

    Simple diffusion: End-to- end diffusion for high resolution images, 2023

    Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. Simple diffusion: End-to- end diffusion for high resolution images, 2023

    2023

  58. [66]

    On the limitations of compute thresholds as a governance strategy, 2024

    Sara Hooker. On the limitations of compute thresholds as a governance strategy, 2024

    2024

  59. [67]

    Multilayer feedforward networks are universal approximators

    Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359–366, 1989. 131

    1989

  60. [68]

    Local relation networks for image recognition

    Han Hu, Zheng Zhang, Zhenda Xie, and Stephen Lin. Local relation networks for image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , 2019

    2019

  61. [69]

    Squeeze-and-excitation networks

    Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018

    2018

  62. [70]

    Arbitrary style transfer in real-time with adaptive instance normalization

    Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2017

    2017

  63. [71]

    Lawrence Zitnick

    Drew A Hudson and C. Lawrence Zitnick. Generative adversarial transformers. Proceedings of the 38th International Conference on Machine Learning, ICML 2021 , 2021

    2021

  64. [72]

    Lawrence Zitnick

    Drew A Hudson and C. Lawrence Zitnick. Compositional transformers for scene generation. Advances in Neural Information Processing Systems NeurIPS 2021 , 2021

    2021

  65. [73]

    Estimation of Non-Normalized Statistical Models by Score Matching

    Aapo Hyv¨ arinen. Estimation of Non-Normalized Statistical Models by Score Matching. Journal of Machine Learning Research , 6(24):695–709, 2005

    2005

  66. [74]

    Averaging weights leads to wider optima and better generalization

    Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. arXiv preprint arXiv:1803.05407 , 2018

    2018 arXiv

  67. [75]

    Oneformer: One transformer to rule universal image segmentation

    Jitesh Jain, Jiachen Li, Mang Tik Chiu, Ali Hassani, Nikita Orlov, and Humphrey Shi. Oneformer: One transformer to rule universal image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2989–2998, 2023. 132

    2023

  68. [76]

    Semask: Semantically masked transformers for semantic segmentation

    Jitesh Jain, Anukriti Singh, Nikita Orlov, Zilong Huang, Jiachen Li, Steven Walton, and Humphrey Shi. Semask: Semantically masked transformers for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops , pages 752–761, 2023

    2023

  69. [77]

    Face Perception, chapter 43

    Nancy Kanwisher and Galit Yovel. Face Perception, chapter 43. John Wiley & Sons, Ltd, 2009

    2009

  70. [78]

    Progressive growing of GANs for improved quality, stability, and variation

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. In International Conference on Learning Representations, 2018

    2018

  71. [79]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2019

    2019

  72. [80]

    Training generative adversarial networks with limited data

    Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Advances in Neural Information Processing Systems , pages 12104–12114. Curran Associates, Inc., 2020

    2020

  73. [81]

    Analyzing and improving the image quality of stylegan

    Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020

    2020

  74. [82]

    Alias-free generative adversarial networks

    Tero Karras, Miika Aittala, Samuli Laine, Erik H¨ ark¨ onen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. In Advances in Neural Information Processing Systems , pages 852–863. Curran Associates, Inc., 2021. 133

    2021

  75. [83]

    Design amortization for bayesian optimal experimental design

    Noble Kennamer, Steven Walton, and Alexander Ihler. Design amortization for bayesian optimal experimental design. Proceedings of the AAAI Conference on Artificial Intelligence, 37(7):8220–8227, 2023

    2023

  76. [84]

    Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation

    Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and Il-Chul Moon. Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation. In Proceedings of the 39th International Conference on Machine Learning , pages 11201–11...

    2022

  77. [85]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017

    2017

  78. [86]

    Glow: Generative Flow with Invertible 1x1 Convolutions

    Durk P Kingma and Prafulla Dhariwal. Glow: Generative Flow with Invertible 1x1 Convolutions. In Advances in Neural Information Processing Systems . Curran Associates, Inc., 2018

    2018

  79. [87]

    Kingma and Ruiqi Gao

    Diederik P. Kingma and Ruiqi Gao. Understanding diffusion objectives as the elbo with simple data augmentation, 2023

    2023

  80. [88]

    Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling

    Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improving variational inference with inverse autoregressive flow, 2017

    2017

  81. [89]

    Big transfer (bit): General visual representation learning, 2020

    Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning, 2020

    2020

  82. [90]

    Revealing the dark secrets of BERT

    Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. Revealing the dark secrets of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP...

    2019

  83. [91]

    Learning multiple layers of features from tiny images.(2009), 2009

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.(2009), 2009

    2009

  84. [92]

    The role of imagenet classes in fr´ echet inception distance

    Tuomas Kynk¨ a¨ anniemi, Tero Karras, Miika Aittala, Timo Aila, and Jaakko Lehtinen. The role of imagenet classes in fr´ echet inception distance. In The Eleventh International Conference on Learning Representations , 2023

    2023

  85. [93]

    Howard, Wayne Hubbard, and Lawrence Jackel

    Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, R. Howard, Wayne Hubbard, and Lawrence Jackel. Handwritten digit recognition with a back- propagation network. In Advances in Neural Information Processing Systems . Morgan-Kaufmann, 1989

    1989

  86. [94]

    Gradient-based learning applied to document recognition

    Yann LeCun, L´ eon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278– 2324, 2002

    2002

  87. [95]

    Smooth manifolds

    John M Lee and John M Lee. Smooth manifolds. Springer, 2003

    2003

  88. [96]

    ViTGAN: Training GANs with vision transformers

    Kwonjoon Lee, Huiwen Chang, Lu Jiang, Han Zhang, Zhuowen Tu, and Ce Liu. ViTGAN: Training GANs with vision transformers. In International Conference on Learning Representations, 2022

    2022

  89. [97]

    SNIP: SINGLE- SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY

    Namhoon Lee, Thalaiyasingam Ajanthan, and Philip Torr. SNIP: SINGLE- SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY. In International Conference on Learning Representations , 2019

    2019

  90. [98]

    xformers: A modular and hackable transformer modelling library

    Benjamin Lefaudeux, Francisco Massa, Diana Liskovich, Wenhan Xiong, Vittorio Caggiano, Sean Naren, Min Xu, Jieru Hu, Marta Tintore, Susan Zhang, Patrick Labatut, Daniel Haziza, Luca Wehrstedt, Jeremy Reizenstein, and Grigory Sizov. xformers: A modular and hackable transformer ...

    2022

  91. [99]

    Demographic bias effects on face image synthesis

    Roberto Leyva, Victor Sanchez, Gregory Epiphaniou, and Carsten Maple. Demographic bias effects on face image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , pages 3818–3826, 2024

    2024

  92. [100]

    Visualizing the loss landscape of neural nets

    Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2018

    2018

  93. [101]

    Learning question classifiers

    Xin Li and Dan Roth. Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics , 2002

    2002

  94. [102]

    M. Lichman. Uci machine learning repository, 2013

    2013

  95. [103]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014

    2014

  96. [104]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023

    2023

  97. [105]

    Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky T. Q. Chen, David Lopez-Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code, 2024

    2024

  98. [106]

    Accelerate tarflow sampling with gs-jacobi iteration, 2025

    Ben Liu and Zhen Qin. Accelerate tarflow sampling with gs-jacobi iteration, 2025

    2025

  99. [107]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023. 136

    2023

  100. [108]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) , 2015

    2015

  101. [109]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pages 10012–10022, 2021

    2021

  102. [110]

    Swin transformer v2: Scaling up capacity and resolution

    Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, and Baining Guo. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (...

    2022

  103. [111]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022

    2022

  104. [112]

    Kan: Kolmogorov-arnold networks

    Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljaˇ ci´ c, Thomas Y Hou, and Max Tegmark. Kan: Kolmogorov-arnold networks. arXiv preprint arXiv:2404.19756 , 2024

    2024 arXiv

  105. [113]

    Decoupled weight decay regularization, 2019

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019

    2019

  106. [114]

    Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. Effective approaches to attention-based neural machine translation, 2015

    2015

  107. [115]

    Macow: Masked convolutional generative flow

    Xuezhe Ma, Xiang Kong, Shanghang Zhang, and Eduard Hovy. Macow: Masked convolutional generative flow. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2019

    2019

  108. [116]

    Learning word vectors for sentiment analysis

    Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings 137 of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142–150, 2011

    2011

  109. [117]

    Martin, C

    D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int’l Conf. Computer Vision , pages 416–423, 2001

    2001

  110. [118]

    Which training methods for GANs do actually converge? In Proceedings of the 35th International Conference on Machine Learning , pages 3481–3490

    Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for GANs do actually converge? In Proceedings of the 35th International Conference on Machine Learning , pages 3481–3490. PMLR, 2018

    2018

  111. [119]

    Are sixteen heads really better than one? In Advances in Neural Information Processing Systems

    Paul Michel, Omer Levy, and Graham Neubig. Are sixteen heads really better than one? In Advances in Neural Information Processing Systems . Curran Associates, Inc., 2019

    2019

  112. [120]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics and Image Processing , pages 722–729, 2008

    2008

  113. [121]

    Masked autoregressive flow for density estimation, 2017

    George Papamakarios, Theo Pavlakou, and Iain Murray. Masked autoregressive flow for density estimation, 2017

    2017

  114. [122]

    On aliased resizing and surprising subtleties in gan evaluation

    Gaurav Parmar, Richard Zhang, and Jun-Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 11410–11420, 2022

    2022

  115. [123]

    Image transformer

    Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and Dustin Tran. Image transformer. In International Conference on Machine Learning (ICML) , 2018. 138

    2018

  116. [124]

    Pytorch: An imperative style, high-performance deep learning library, 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K¨ opf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu F...

    2019

  117. [125]

    GloVe: Global vectors for word representation

    Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 1532–1543, Doha, Qatar,

    2014

  118. [126]

    B. T. Polyak and A. B. Juditsky. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization , 30(4):838–855, 1992

    1992

  119. [127]

    Designing network design spaces

    Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollar. Designing network design spaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2020

    2020

  120. [128]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020

    2020

  121. [129]

    Stand-alone self-attention in vision models

    Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello, Anselm Levskaya, and Jon Shlens. Stand-alone self-attention in vision models. In Advances in Neural Information Processing Systems . Curran Associates, Inc., 2019

    2019

  122. [130]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨ orn Ommer. High-resolution image synthesis with latent diffusion models. In 139 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022

    2022

  123. [131]

    Projected gans converge faster

    Axel Sauer, Kashyap Chitta, Jens M¨ uller, and Andreas Geiger. Projected gans converge faster. In Advances in Neural Information Processing Systems , pages 17480–17492. Curran Associates, Inc., 2021

    2021

  124. [132]

    Stylegan-xl: Scaling stylegan to large diverse datasets

    Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pages 1–10, 2022

    2022

  125. [133]

    Are emergent abilities of large language models a mirage? In Advances in Neural Information Processing Systems, pages 55565–55581

    Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are emergent abilities of large language models a mirage? In Advances in Neural Information Processing Systems, pages 55565–55581. Curran Associates, Inc., 2023

    2023

  126. [134]

    Jermyn, Joe Benton, and Buck Shlegeris

    Adam Scherlis, Kshitij Sachan, Adam S. Jermyn, Joe Benton, and Buck Shlegeris. Polysemanticity and capacity in neural networks. CoRR, abs/2210.01892, 2022

    2022 arXiv

  127. [135]

    Neural machine translation of rare words with subword units

    Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 1715– 1725, Berlin, Germany, 2016. Associat...

    2016

  128. [136]

    Flashattention-3: Fast and accurate attention with asynchrony and low- precision

    Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention-3: Fast and accurate attention with asynchrony and low- precision. arXiv preprint arXiv:2407.08608 , 2024

    2024 arXiv

  129. [137]

    Deep inside convolutional networks: Visualising image classification models and saliency maps, 2014

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps, 2014. 140

    2014

  130. [138]

    Recursive deep models for semantic compositionality over a sentiment treebank

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language proce...

    2013

  131. [139]

    Stanley, David B

    Kenneth O. Stanley, David B. D’Ambrosio, and Jason Gauci. A hypercube-based encoding for evolving large-scale neural networks. Artificial Life, 15(2):185–212, 2009

    2009

  132. [140]

    Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L

    George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L. Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. ...

    2023

  133. [141]

    Revisiting unreasonable effectiveness of data in deep learning era

    Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2017

    2017

  134. [142]

    The bitter lesson, 2019

    Richard Sutton. The bitter lesson, 2019

    2019

  135. [143]

    Rethinking the inception architecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016

    2016

  136. [144]

    SAN: Inducing metrizability of GAN with discriminative normalized linear layer

    Yuhta Takida, Masaaki Imaizumi, Takashi Shibuya, Chieh-Hsin Lai, Toshimitsu Uesaka, Naoki Murata, and Yuki Mitsufuji. SAN: Inducing metrizability of GAN with discriminative normalized linear layer. In The Twelfth International Conference on Learning Representations, 2024. 141

    2024

  137. [145]

    An analysis of attention mechanisms: The case of word sense disambiguation in neural machine translation

    Gongbo Tang, Rico Sennrich, and Joakim Nivre. An analysis of attention mechanisms: The case of word sense disambiguation in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers , pages 26–35, Brussels, Belgium, 2018. Associ...

    2018

  138. [146]

    Relay diffusion: Unifying diffusion process across resolutions for image synthesis, 2023

    Jiayan Teng, Wendi Zheng, Ming Ding, Wenyi Hong, Jianqiao Wangni, Zhuoyi Yang, and Jie Tang. Relay diffusion: Unifying diffusion process across resolutions for image synthesis, 2023

    2023

  139. [147]

    Antonio Torralba, Rob Fergus, and William T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 30(11):1958–1970, 2008

    1958

  140. [148]

    Fixing the train- test resolution discrepancy

    Hugo Touvron, Andrea Vedaldi, Matthijs Douze, and Herve Jegou. Fixing the train- test resolution discrepancy. In Advances in Neural Information Processing Systems . Curran Associates, Inc., 2019

    2019

  141. [149]

    Training data-efficient image transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers & distillation through attention. In Proceedings of the 38th International Conference on Machine Learning , pages 10347–10357. PMLR, 2021

    2021

  142. [150]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017

    2017

  143. [151]

    Scaling local self-attention for parameter efficient visual backbones, 2021

    Ashish Vaswani, Prajit Ramachandran, Aravind Srinivas, Niki Parmar, Blake Hechtman, and Jonathon Shlens. Scaling local self-attention for parameter efficient visual backbones, 2021. 142

    2021

  144. [152]

    A 0-shot self-attention mechanism for accelerated diagonal attention

    Mario Viti, Nadiya Shvai, Arcadi Llanza, and Amir Nakib. A 0-shot self-attention mechanism for accelerated diagonal attention. In Proceedings of the Winter Conference on Applications of Computer Vision (WACV) , pages 7308–7315, 2025

    2025

  145. [153]

    Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned

    Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 5797...

    2019

  146. [154]

    Isomorphism, normalizing flows, and density estimation: Preserving relationships between data, 2022

    Steven Walton. Isomorphism, normalizing flows, and density estimation: Preserving relationships between data, 2022

    2022

  147. [155]

    Training compact transformers from scratch in 30 minutes with pytorch

    Steven Walton, Ali Hassani, Abulikemu Abuduweili, and Humphrey Shi. Training compact transformers from scratch in 30 minutes with pytorch. medium.com/pytorch, 2021

    2021

  148. [156]

    Efficient image generation with variadic attention heads

    Steven Walton, Ali Hassani, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Efficient image generation with variadic attention heads. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025

    2025

  149. [157]

    Distilling normalizing flows

    Steven Walton, Valeriy Klyukin, Maksim Artemev, Denis Derkach, Nikita Orlov, and Humphrey Shi. Distilling normalizing flows. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , 2025

    2025

  150. [158]

    Diffusion-GAN: Training GANs with diffusion

    Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou. Diffusion-GAN: Training GANs with diffusion. In The Eleventh International Conference on Learning Representations, 2023. 143

    2023

  151. [159]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models. T...

    2022

  152. [160]

    Pytorch image models, 2019

    Ross Wightman. Pytorch image models, 2019

    2019

  153. [161]

    Resnet strikes back: An improved training procedure in timm, 2021

    Ross Wightman, Hugo Touvron, and Herv´ e J´ egou. Resnet strikes back: An improved training procedure in timm, 2021

    2021

  154. [162]

    Convnext v2: Co-designing and scaling convnets with masked autoencoders

    Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1613...

    2023

  155. [163]

    Efficient streaming language models with attention sinks, 2024

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks, 2024

    2024

  156. [164]

    Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017

    Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017

    2017

  157. [165]

    Data-efficient instance generation from instance discrimination

    Ceyuan Yang, Yujun Shen, Yinghao Xu, and Bolei Zhou. Data-efficient instance generation from instance discrimination. In Advances in Neural Information Processing Systems, pages 9378–9390. Curran Associates, Inc., 2021

    2021

  158. [166]

    Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop

    Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 , 2015

    2015 arXiv

  159. [167]

    Cutmix: Regularization strategy to train strong classifiers with 144 localizable features

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with 144 localizable features. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 6023–6032, 2019

    2019

  160. [168]

    Normalizing flows are capable generative models, 2024

    Shuangfei Zhai, Ruixiang Zhang, Preetum Nakkiran, David Berthelot, Jiatao Gu, Huangjie Zheng, Tianrong Chen, Miguel Angel Bautista, Navdeep Jaitly, and Josh Susskind. Normalizing flows are capable generative models, 2024

    2024

  161. [169]

    Styleswin: Transformer-based gan for high-resolution image generation

    Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang, and Baining Guo. Styleswin: Transformer-based gan for high-resolution image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 11304–11314, 2022

    2022

  162. [170]

    mixup: Beyond empirical risk minimization

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 , 2017

    2017 arXiv

  163. [171]

    Self-attention generative adversarial networks

    Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. Self-attention generative adversarial networks. In Proceedings of the 36th International Conference on Machine Learning , pages 7354–7363. PMLR, 2019

    2019

  164. [172]

    Character-level convolutional networks for text classification

    Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. arXiv preprint arXiv:1509.01626 , 2015

    2015 arXiv

  165. [173]

    Nested hierarchical transformer: Towards accurate, data-efficient and interpretable visual understanding

    Zizhao Zhang, Han Zhang, Long Zhao, Ting Chen, Sercan ¨O Arik, and Tomas Pfister. Nested hierarchical transformer: Towards accurate, data-efficient and interpretable visual understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 3417–3425, 2022

    2022

  166. [174]

    Improved transformer for high-resolution gans

    Long Zhao, Zizhao Zhang, Ting Chen, Dimitris Metaxas, and Han Zhang. Improved transformer for high-resolution gans. In Advances in Neural Information Processing Systems, pages 18367–18380. Curran Associates, Inc., 2021. 145

    2021

  167. [175]

    Improved consistency regularization for gans

    Zhengli Zhao, Sameer Singh, Honglak Lee, Zizhao Zhang, Augustus Odena, and Han Zhang. Improved consistency regularization for gans. In Proceedings of the AAAI conference on artificial intelligence , pages 11033–11041, 2021

    2021

  168. [176]

    Fast training of diffusion models with masked transformers

    Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers. In Transactions on Machine Learning Research (TMLR), 2024

    2024

  169. [177]

    Zheng, Tong Xu, and Enhong Chen

    Hui Zhong, Zaiyi Chen, Chuan Qin, Zai Huang, Vincent W. Zheng, Tong Xu, and Enhong Chen. Adam revisited: a weighted past gradients perspective. Frontiers of Computer Science, 14(5), 2020

    2020

  170. [178]

    Random erasing data augmentation

    Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 13001–13008, 2020

    2020

  171. [179]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In CVPR, 2017

    2017

  172. [180]

    Unpaired image-to- image translation using cycle-consistent adversarial networks

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to- image translation using cycle-consistent adversarial networks. In Computer Vision (ICCV), 2017 IEEE International Conference on , 2017. 146

    2017

  173. [2014]

    Association for Computational Linguistics

  174. [2022]

    https://transformer-circuits.pub/2022/solu/index.html

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.