Pith. sign in

REVIEW 4 major objections 6 minor 59 references

A Decade of Deep Learning: A Survey on The Magnificent Seven

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This survey argues that seven algorithms—ResNets, Transformers, GANs, VAEs, GNNs, CLIP, and diffusion models—are the most influential deep learning developments of 2013–2024.

desk verdict A survey with a broken selection method and integrity problems that undercut its central claim. read the letter →

arxiv 2412.16188 v1 pith:2WY4INNK submitted 2024-12-13 cs.LG cs.AI

Dilshod Azizov , Muhammad Arslan Manzoor , Velibor Bojkovic , Yingxu Wang , Zixiao Wang , Zangir Iklassov , Kailong Zhao , Liang Li
show 4 more authors
Siwei Liu Yu Zhong Wei Liu Shangsong Liang
This is my paper · ORCID
classification cs.LGcs.AI MSC 68T07
keywords deeplearningsurveyResNetsTransformersGANsVAEsGNNsCLIPdiffusionmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey argues that seven algorithms—ResNets, Transformers, GANs, VAEs, GNNs, CLIP, and diffusion models—are the most influential deep learning developments of 2013–2024. It selects them through a questionnaire answered by 100 academics and professionals, keeps only algorithms born within the decade, and orders them by the citation counts of their foundational papers as of December 10, 2024. The authors claim this is the first survey to treat these seven as a group and intend it as a practical manual for newcomers and researchers transitioning into deep learning. If the selection is accepted, the paper provides a compact entry point to the architectures that now dominate vision, language, and generative modeling.

What carries the argument

The load-bearing mechanism is the two-stage selection protocol: a four-question survey distributed to students, research assistants, postdocs, and faculty yields an initial candidate set; a chronological filter restricts it to algorithms introduced within 2013–2024; and citation counts of each algorithm's foundational paper, taken on December 10, 2024, set the final order. This protocol turns subjective judgments of influence into an ordered, reproducible list, and the composition of the respondent pool determines whether that list generalizes.

What would settle it

Run the same four-question survey with a much larger and more globally diverse respondent pool and re-collect citation counts at a later date; if a different set of algorithms reaches the top seven, or the ordering changes materially, the paper's central ranking fails as a stable description. A simpler check is to compute citation counts for the excluded algorithms (CNN, LSTM) within the same window; if those counts dwarf several of the seven, the decade filter rather than the citation proxy is doing the selecting.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central result is a grouping and a ranking: ResNets, Transformers, GANs, VAEs, GNNs, CLIP, and diffusion models are 'The Magnificent Seven' of the past decade. The ranking starts from a survey of 100 respondents; twelve candidate algorithms survive aggregation, and five are dropped because they predate the 2013–2024 window (CNN and LSTM are named). The final order follows the citation counts of each algorithm's foundational paper, with ResNets first (247,440), Transformers second (144,501), GANs third (75,445), VAEs fourth (40,549), GNNs fifth (40,130), CLIP sixth (25,847), and diffusion models seventh (15,632). Each chapter then presents the algorithm's core architecture, mathematical foundations, algorithmic procedure, training and optimization, extensions, applications, challenges, and future directions, all in service of the stated goal: a practical manual for entering deep learning.

Load-bearing premise

The ranking is only as trustworthy as the respondent pool and the citation snapshot: if the 100 respondents, drawn mostly from two universities, are not representative of the global deep learning community, the 'Magnificent Seven' may not be the true top seven.

Editorial extensions

If this is right

  • If the ranking is accepted, newcomers have a defensible shortlist for what to learn first: the seven architectures that define modern practice.
  • The citation ordering provides a rough influence hierarchy, placing ResNets highest and diffusion models lowest among the seven despite diffusion's recent dominance.
  • Each algorithm's chapter supplies a structured reference covering math, training, variants, applications, and challenges, usable independent of the ranking.
  • The decade boundary means older workhorses like CNN and LSTM are excluded by design, not because the authors consider them unimportant.
  • The paper's future-work agenda anticipates expanding the list and building new evaluation measures and benchmarks for deep learning algorithms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Re-running the same survey with a larger, more geographically diverse respondent pool is the natural test of whether these seven, and not another seven, are the decade's leaders.
  • Because citation counts accumulate over time, the ranking may understate the current influence of recent algorithms such as CLIP and diffusion models; a usage-based or survey-based recency measure would likely shift the order.
  • The two-stage protocol itself could be reused as a standing method to rank algorithmic influence in future decades.
  • If the selection is right, textbooks and course syllabi for the decade could reasonably be organized around these seven architectures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This manuscript proposes a survey of what it calls the "Magnificent Seven" deep learning algorithms — ResNets, Transformers, GANs, VAEs, GNNs, CLIP, and Diffusion Models — selected through a questionnaire completed by 100 respondents and ranked by Google Scholar citation counts of foundational papers. For each algorithm, the paper provides an overview, core architecture, mathematical foundations, algorithmic procedure, training and optimization, extensions, applications, challenges, and future directions, plus a discussion of common building blocks. The stated aim is to serve as a practical manual for newcomers and transitioning researchers.

Significance. The paper is clearly written in a tutorial style and covers a broad set of important material, including the main equations for each architecture (e.g., Eqs. (1), (3), (5), (6), (7), (8), (9)) and practical training advice. If the selection were robust, the survey would be a convenient entry point. However, the central selection claim is undermined by an internal chronological contradiction and by a survey base that the authors themselves acknowledge is institutionally concentrated. The paper is also not a systematic review; it is a curated tutorial, and the "top seven" framing overstates the evidence. The manuscript ships no machine-checked proofs or reproducible code, which is not required for a survey, but it does offer a useful consolidated reference for the included architectures.

major comments (4)
  1. [Section 3.1 / Footnote 1 / Section 4.5] The selection criterion in Section 3.1 removes CNN and LSTM because their chronological beginnings lie outside 2013–2024, and Footnote 1 explicitly dates GNNs to 2016. Yet Section 4.5 (Overview) states that GNNs were "introduced in the early 2000s," and the citation-based ranking uses the 2016 GCN paper as the foundational GNN reference. If the chronological-beginnings test is applied uniformly, GNNs should be removed by the same reasoning used to remove CNN and LSTM. The paper either needs to define the criterion as "the breakthrough paper appeared in the window," which would also affect other choices, or it needs a separate argument for why GNNs are included. As written, the selection of the seven is internally inconsistent.
  2. [Section 4.1, Figure 3] Figure 3 is not an original figure. It contains text from another paper on ResNet training heuristics, including "Model Tweaks," "Table 4," references to ResNet-B/C/D, and a citation to "He et al. [9]" that are not part of the present manuscript. This is a copy-paste error that must be corrected; as published it undermines the reliability of the other figures and the survey as a whole.
  3. [Section 3.1 and Bias section] The survey-based selection in Section 3.1 is not as robust as the paper claims. The sample is 100 respondents, and Figure 2a shows a majority from MBZUAI and Sun Yat-sen University, with most respondents being M.Sc./Ph.D. students with 1–5 years of experience. The paper provides no response rate, no sampling frame, no independent validation, and the authors' own "Bias" section concedes institutional bias. The Google Scholar citation ranking is also not independent of popularity and includes self-citations. Since the "top seven" claim rests on this methodology, the paper should either report a more diverse and documented survey or reframe the selection as the authors' curated choice informed by the survey.
  4. [Figure 1 caption] Figure 1's caption states that the answers to the survey questions were "randomly generated by the authors." If these are illustrative examples, the caption should say so explicitly and should not present them as actual survey responses; if they are actual responses, the caption is false. In either case, the figure as presented confuses the evidentiary status of the survey. Additionally, the questionnaire itself is not described in the body text beyond this figure, so the reader cannot evaluate the wording or the response format.
minor comments (6)
  1. [Section 2] The references "Smith & Lee (2020)" and "Doe & Row (2021)" appear to be placeholder citations; they are not identifiable real works and should be replaced or removed.
  2. [Section 4.2] The text contains typos "GPT-4o14" and "GPT4o1" for GPT-4o; these should be corrected for consistency.
  3. [Section 4.4] The citation to "Dziugaite (2020)" for layer normalization in VAEs is not a standard reference for this technique; consider citing the original layer normalization work instead.
  4. [Section 5.2] The statement that GANs "Employ RMSProp, ADAM, and rate scheduling" is vague and unsupported by a specific reference; the sentence should be clarified or removed.
  5. [Section 4.1] The algorithmic procedure for ResNets has a numbering error: item "2. Residual Block Processing:" is immediately followed by "3. Convolutional Layers," leaving an empty step; the numbering should be fixed.
  6. [Figure 2] The three panels of Figure 2 do not report sample sizes per category; adding exact numbers would help the reader assess the representativeness of the survey.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the 'Magnificent Seven' selection is an externally anchored survey-plus-citation ranking, not a derivation that reduces to its own inputs; the GNN dating inconsistency is a consistency flaw, not circularity.

full rationale

The paper does not derive predictions from first principles; it selects seven algorithms via a reported survey and then ranks them by Google Scholar citation counts. The selection chain is explicit and external: 'Following the aggregation of responses, the top 12 algorithms were determined based on the collective vote of the participants' and 'The research impact of each algorithm was measured using the citation score of the seminal articles.' No parameter is fitted and then renamed a prediction, and no equation is defined in terms of the target conclusion. The presence of many self-citations by the authors in application and variant sections is not load-bearing for the central claim, which rests on survey responses and external citation counts of foundational papers by other authors. The paper's own Bias section acknowledges that respondents came mostly from a limited number of universities, and this is a legitimate limitation of the survey's generalizability, but it is not a circularity: the survey result is an empirical input, not a conclusion forced by definition. The internal inconsistency between Footnote 1 dating GNNs to 2016 and Section 4.5 stating GNNs were 'introduced in the early 2000s' undermines the consistency of the chronological filter, but it is a correctness and coherence flaw, not a circular reduction: the paper does not define GNN influence in terms of the survey or of its own previous work. Accordingly, no circular step meets the required standard of quoting a specific reduction by construction.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper's central ranking rests on assumptions about the validity of Google Scholar citations and the representativeness of a small, locally sourced survey. No free parameters or invented entities are introduced because the paper presents no new models or derivations.

assumptions (3)
  • domain assumption Google Scholar citation counts as of December 10, 2024 are a valid proxy for the influence of an algorithm.
    The paper uses these counts to rank the seven algorithms in Section 3.1, without discussing limitations such as citation inflation, self-citations, or the choice of the specific foundational paper for each algorithm.
  • domain assumption The 100 survey respondents, mostly from MBZUAI and Sun Yat-sen University, are representative of the global deep learning community.
    Section 3.1 and Figure 2 show the respondents are concentrated in two institutions and are mostly M.Sc. and Ph.D. students, which limits the generality of their preferences.
  • domain assumption The chosen foundational papers, e.g., Kipf & Welling (2016) for GNNs and Ho et al. (2020) for diffusion models, are the correct canonical references within the 2013-2024 window.
    For GNNs, the paper claims introduction in the early 2000s but ranks based on a 2016 paper; for diffusion, it uses Ho et al. (2020) rather than Sohl-Dickstein et al. (2015). This choice affects the citation ranking.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Decade of Deep Learning: A Survey on The Magnificent Seven." pith.science (2026). https://pith.science/paper/2WY4INNK

@misc{pith2026241216188,
  author       = {Pith},
  title        = {Pith review of: A Decade of Deep Learning: A Survey on The Magnificent Seven},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2WY4INNK}},
  note         = {Machine review of arXiv:2412.16188}
}
read the original abstract

Deep learning has fundamentally reshaped the landscape of artificial intelligence over the past decade, enabling remarkable achievements across diverse domains. At the heart of these developments lie multi-layered neural network architectures that excel at automatic feature extraction, leading to significant improvements in machine learning tasks. To demystify these advances and offer accessible guidance, we present a comprehensive overview of the most influential deep learning algorithms selected through a broad-based survey of the field. Our discussion centers on pivotal architectures, including Residual Networks, Transformers, Generative Adversarial Networks, Variational Autoencoders, Graph Neural Networks, Contrastive Language-Image Pre-training, and Diffusion models. We detail their historical context, highlight their mathematical foundations and algorithmic principles, and examine subsequent variants, extensions, and practical considerations such as training methodologies, normalization techniques, and learning rate schedules. Beyond historical and technical insights, we also address their applications, challenges, and potential research directions. This survey aims to serve as a practical manual for both newcomers seeking an entry point into cutting-edge deep learning methods and experienced researchers transitioning into this rapidly evolving domain.

Figures

Figures reproduced from arXiv: 2412.16188 by the authors.

Figure 1
Figure 1. Examples of the questions provided to respondents, along with answers. The answers to the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Distribution of respondents in our survey questionnaire. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 2
Figure 2. Three ResNet tweaks. ResNet-B modifies the with identity mappings, ks such as image classification, object detection, [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Illustration of the transformer architecture with encoder and decoder: the en￾coder processes input sequences, while the decoder generates out￾puts using self-attention, cross￾attention, and feedforward lay￾ers. Core Architecture. The transformer model eschews traditio…
Figure 5
Figure 5. Figure 5: The GANs core architecture involves two neural networks: a generator creating fake data, and a [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Illustration of VAEs core architecture which consist of an encoder mapping data to latent space [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Demonstration of GNNs core architecture where model takes a graph as input (nodes, edges, [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: CLIP core architecture processes paired image and text inputs through separate encoders, aligning [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Illustration of diffusion model core architecture where it takes data as input, progressively adds [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 7 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  2. [5]

    Relational inductive biases, deep learning, and graph networks.arXiv preprint arXiv:1806.01261,

    Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks.arXiv preprint arXiv:1806.01261,

    arXiv

  3. [7]

    Gener- ating sentences from a continuous space.arXiv preprint arXiv:1511.06349,

    Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. Gener- ating sentences from a continuous space.arXiv preprint arXiv:1511.06349,

    arXiv

  4. [11]

    Rethinking attention with performers

    Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. Rethinking attention with performers. arXiv preprint arXiv:2009.14794,

    arXiv 2009

  5. [13]

    Bert: Pre-trainingofdeepbidirectional transformers for language understanding

    JacobDevlin, Ming-WeiChang, KentonLee, andKristinaToutanova. Bert: Pre-trainingofdeepbidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,

    work page 2019

  6. [14]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    arXiv

  7. [15]

    Can: Creative adversar- ial networks, generating" art" by learning about styles and deviating from style norms.arXiv preprint arXiv:1706.07068,

    Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, and Marian Mazzone. Can: Creative adversar- ial networks, generating" art" by learning about styles and deviating from style norms.arXiv preprint arXiv:1706.07068,

    arXiv

  8. [17]

    Dynamical variational autoencoders: A comprehensive review.arXiv preprint arXiv:2008.12595,

    Laurent Girin, Simon Leglaive, Xiaoyu Bie, Julien Diard, Thomas Hueber, and Xavier Alameda-Pineda. Dynamical variational autoencoders: A comprehensive review.arXiv preprint arXiv:2008.12595,

    arXiv 2008

Show all 59 references
  1. [20]

    Perceiver: General perception with iterative attention.arXiv preprint arXiv:2103.03206,

    Andrew Jaegle, Francisco Gimeno, Andrew Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention.arXiv preprint arXiv:2103.03206,

    arXiv

  2. [21]

    Scaling up visual and vision-language repre- sentation learning with noisy text supervision.arXiv preprint arXiv:2102.05918,

    Robin Jia, Percy Liang, Aditi Raghunathan, and Nic Moritz. Scaling up visual and vision-language repre- sentation learning with noisy text supervision.arXiv preprint arXiv:2102.05918,

    arXiv

  3. [22]

    Torsional diffusion for molecular conformer generation.arXiv preprint arXiv:2107.07660,

    Bowen Jing, Stephan Eismann, Patricia Suriana, Greg Chirikjian, and Regina Barzilay. Torsional diffusion for molecular conformer generation.arXiv preprint arXiv:2107.07660,

    arXiv

  4. [23]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

    arXiv

  5. [24]

    Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114,

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114,

    arXiv

  6. [25]

    Variational diffusion models.arXiv preprint arXiv:2107.00630,

    Durk P Kingma and Tim Salimans. Variational diffusion models.arXiv preprint arXiv:2107.00630,

    arXiv

  7. [26]

    Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907,

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907,

    arXiv

  8. [30]

    Multimodality representation learning: A survey on evolution, pretraining and its applications

    Muhammad Arslan Manzoor, Sarah Albarri, Ziting Xian, Zaiqiao Meng, Preslav Nakov, and Shangsong Liang. Multimodality representation learning: A survey on evolution, pretraining and its applications. ACM Trans. Multimedia Comput. Commun. Appl., 20(3), oct2023. ISSN1551-6857. do...

    doi

  9. [32]

    Mixed precision training.arXiv preprint arXiv:1710.03740,

    Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training.arXiv preprint arXiv:1710.03740,

    arXiv

  10. [34]

    An image is worth more than 16x16 patches: Exploring transformers on individual pixels.arXiv preprint arXiv:2406.09415,

    Duy-Kien Nguyen, Mahmoud Assran, Unnat Jain, Martin R Oswald, Cees GM Snoek, and Xinlei Chen. An image is worth more than 16x16 patches: Exploring transformers on individual pixels.arXiv preprint arXiv:2406.09415,

    arXiv

  11. [35]

    Styleclip: Text-driven manipulation of stylegan imagery.arXiv preprint arXiv:2103.17249,

    Oran Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. Styleclip: Text-driven manipulation of stylegan imagery.arXiv preprint arXiv:2103.17249,

    arXiv

  12. [36]

    Unsupervised representation learning with deep convolu- tional generative adversarial networks.arXiv preprint arXiv:1511.06434,

    Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolu- tional generative adversarial networks.arXiv preprint arXiv:1511.06434,

    arXiv

  13. [37]

    Under Review

    32 Preprint. Under Review. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sharan Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, and Jack Clark. Whisper: Better speech recognition through large-scale weak supervision. arXiv preprint arXiv:2212.03591,

    arXiv

  14. [39]

    Prajit Ramachandran, Barret Zoph, and Quoc V. Le. Deep multimodal fusion. arXiv preprint arXiv:1707.07262,

    arXiv

  15. [40]

    Zero-shot text-to-image generation.arXiv preprint arXiv:2102.12092,

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation.arXiv preprint arXiv:2102.12092,

    arXiv

  16. [41]

    Stochasticbackpropagationandapproximate inference in deep generative models.arXiv preprint arXiv:1401.4082,

    DaniloJimenezRezende, ShakirMohamed, andDaanWierstra. Stochasticbackpropagationandapproximate inference in deep generative models.arXiv preprint arXiv:1401.4082,

    arXiv

  17. [42]

    Donti, Lynn H

    David Rolnick, Priya L. Donti, Lynn H. Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, An- drew Slavin Ross, Nikola Milojevic-Dupont, Natasha Jaques, Anna Waldman-Brown, et al. Tackling climate change with machine learning.arXiv preprint arXiv:1906.05433,

    1906 arXiv

  18. [43]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

    arXiv

  19. [44]

    Rethinking inter- pretability in the era of large language models.arXiv preprint arXiv:2402.01761,

    Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Rethinking inter- pretability in the era of large language models.arXiv preprint arXiv:2402.01761,

    arXiv

  20. [45]

    Deepunsupervisedlearning using nonequilibrium thermodynamics.arXiv preprint arXiv:1503.03585,

    JaschaSohl-Dickstein, EricAWeiss, NiruMaheswaranathan, andSuryaGanguli. Deepunsupervisedlearning using nonequilibrium thermodynamics.arXiv preprint arXiv:1503.03585,

    arXiv

  21. [46]

    Score-based generative modeling through stochastic differential equations

    Yang Song and Stefano Ermon. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456,

    2011 arXiv

  22. [47]

    The carbon footprint of training large transformer models

    Emma Strubell, Ananya Ganesh, and Andrew McCallum. The carbon footprint of training large transformer models. arXiv preprint arXiv:1906.02243,

    1906 arXiv

  23. [49]

    Vae with a vampprior.arXiv preprint arXiv:1705.07120,

    Jakub M Tomczak and Max Welling. Vae with a vampprior.arXiv preprint arXiv:1705.07120,

    arXiv

  24. [50]

    All languages matter: Evaluating lmms on culturally diverse 100 languages.arXiv preprint arXiv:2411.16508,

    Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana, Noor Ahsan, Nevasini Sasikumar, Omkar Thawakar, Henok Biadglign Ademtew, Yahya Hmaiti, Amandeep Kumar, Kartik Kuckreja, et al. All languages matter: Evaluating lmms on culturally diverse 100 languages.arXiv preprint arXiv:2...

    arXiv

  25. [51]

    Graph attention networks.arXiv preprint arXiv:1710.10903,

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks.arXiv preprint arXiv:1710.10903,

    arXiv

  26. [52]

    Li, Madian Khabsa, Han Fang, and Hao Ma

    Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768, 2020b. Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High- resolution image synthesis and...

    2006 arXiv

  27. [53]

    Enhancing conversational recommendation systems with representation fusion.ACM Transactions on the Web, 2023b

    Yingxu Wang, Xiaoru Chen, Jinyuan Fang, Zaiqiao Meng, and Shangsong Liang. Enhancing conversational recommendation systems with representation fusion.ACM Transactions on the Web, 2023b. Yixin Wang and Deva Ramanan. Understanding the behaviors of contrastive losses. arXiv prepr...

    arXiv

  28. [54]

    Codet5+: Opencodelargelanguage models for codeunderstanding andgeneration

    Yue Wang, Hung Le, Akhilesh Deepak Gotmare, Nghi DQ Bui, Junnan Li, and Steven CH Hoi. Codet5+: Opencodelargelanguage models for codeunderstanding andgeneration. arXiv preprint arXiv:2305.07922, 2023c. Douglas Watson, Didrik Hoogeboom, and Max Welling. Learning to efficiently ...

    arXiv

  29. [55]

    Wav2clip: Learning robust audio representations from clip

    Ho-Hsiang Wu, Prem Seetharaman, Kundan Kumar, and Juan Pablo Bello. Wav2clip: Learning robust audio representations from clip. InICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE,

    2022

  30. [56]

    Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications

    Haowen Xu, Wenxiao Chen, Nengwen Zhao, Zeyan Li, Jiahui Bu, Zhihan Li, Ying Liu, Youjian Zhao, Dan Pei, and Yang Feng. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. Proceedings of the 2018 World Wide Web Conference, 2018a. 3...

    2018 arXiv

  31. [57]

    Diffsound: Discrete diffusion model for text-to- speech

    Shihua Yang, Danwei Li, Shan Wang, and Simon King. Diffsound: Discrete diffusion model for text-to- speech. arXiv preprint arXiv:2205.14217, 2022a. Y Yang, M Li, and X Zhou. Diffusion-based image-to-image translation.CVPR, 2022b. Tianjun Yao, Yingxu Wang, Kun Zhang, and Shangs...

    arXiv

  32. [58]

    Da-vegan: Differentiably augmenting vae-gan for microstructure reconstruction from extremely small data sets

    Yichi Zhang, Paul Seibert, Alexandra Otto, Alexander Raßloff, Marreddy Ambati, and Markus Kästner. Da-vegan: Differentiably augmenting vae-gan for microstructure reconstruction from extremely small data sets. Computational Materials Science, 2024a. Yizhuo Zhang, Heng Wang, Sha...

    arXiv

  33. [59]

    Denseclip: Language-guided dense prediction with context-aware prompting.arXiv preprint arXiv:2112.01518,

    Yuhui Zhou, Hao Xu, Linchao Bao, Shuting Geng, Yuning Wei, and Yi Yang. Denseclip: Language-guided dense prediction with context-aware prompting.arXiv preprint arXiv:2112.01518,

    arXiv

  34. [1943]

    L. M. Mescheder, S. Nowozin, and A. Geiger. Numerics of gans.arXiv preprint arXiv:1705.08894,

    arXiv

  35. [1969]

    Mirza and S

    M. Mirza and S. Osindero. Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784,

    arXiv

  36. [2000]

    Neural message passing for quantum chemistry.arXiv preprint arXiv:1704.01212,

    Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry.arXiv preprint arXiv:1704.01212,

    arXiv

  37. [2007]

    Understanding and improving interpolation in autoencoders via an adversarial regularizer.arXiv preprint arXiv:1807.07543,

    David Berthelot, Colin Raffel, Aurko Roy, and Ian Goodfellow. Understanding and improving interpolation in autoencoders via an adversarial regularizer.arXiv preprint arXiv:1807.07543,

    arXiv

  38. [2009]

    Lungren, Kather- ine Shpanskaya, Wynton Dillon, and Curtis P

    Pranav Rajpurkar, Jeremy Irvin, Michaela Bagulayicioglu, Finale Doshi-Velez, Matthew P. Lungren, Kather- ine Shpanskaya, Wynton Dillon, and Curtis P. Langlotz. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning.arXiv preprint arXiv:1711.05225,

    arXiv

  39. [2012]

    A theoretical framework for back-propagation

    Yann LeCun, D Touresky, G Hinton, and T Sejnowski. A theoretical framework for back-propagation. In Proceedings of the 1988 connectionist models summer school,

    1988

  40. [2013]

    Toward trustworthy ai development: Mechanisms for sup- porting verifiable claims.arXiv preprint arXiv:2004.07213,

    Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, et al. Toward trustworthy ai development: Mechanisms for sup- porting verifiable claims.arXiv preprint arXiv:2004.07213,

    2004 arXiv

  41. [2014]

    Draw: A recurrent neural network for image generation.arXiv preprint arXiv:1502.04623,

    Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra. Draw: A recurrent neural network for image generation.arXiv preprint arXiv:1502.04623,

    arXiv

  42. [2015]

    Conformer: Convolution-augmented transformer for speech recog- nition

    Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. Conformer: Convolution-augmented transformer for speech recog- nition. arXiv preprint arXiv:2005.08100,

    2005 arXiv

  43. [2016]

    Reformer: The efficient transformer.arXiv preprint arXiv:2001.04451,

    Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer.arXiv preprint arXiv:2001.04451,

    2001 arXiv

  44. [2017]

    Wasserstein gan.arXiv preprint arXiv:1701.07875,

    Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan.arXiv preprint arXiv:1701.07875,

    arXiv

  45. [2018]

    Rethinking atrous convolution for semantic image segmentation.arXiv preprint arXiv:1706.05587,

    Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation.arXiv preprint arXiv:1706.05587,

    arXiv

  46. [2019]

    Llm-based multi-agent reinforcement learning: Current and future directions.arXiv preprint arXiv:2405.11106,

    Chuanneng Sun, Songjun Huang, and Dario Pompili. Llm-based multi-agent reinforcement learning: Current and future directions.arXiv preprint arXiv:2405.11106,

    arXiv

  47. [2020]

    Spectral networks and locally connected networks on graphs.arXiv preprint arXiv:1312.6203,

    Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs.arXiv preprint arXiv:1312.6203,

    arXiv

  48. [2021]

    Safari: Cross-lingual bias and factuality detection in news media and news articles

    Dilshod Azizov, Zain Mujahid, Hilal AlQuabeh, Preslav Nakov, and Shangsong Liang. Safari: Cross-lingual bias and factuality detection in news media and news articles. InFindings of the Association for Compu- tational Linguistics: EMNLP 2024,

    2024

  49. [2022]

    Evaluating the generalization ability of quantized llms: Benchmark, analysis, and toolbox

    Yijun Liu, Yuan Meng, Fang Wu, Shenhao Peng, Hang Yao, Chaoyu Guan, Chen Tang, Xinzhu Ma, Zhi Wang, and Wenwu Zhu. Evaluating the generalization ability of quantized llms: Benchmark, analysis, and toolbox. arXiv preprint arXiv:2406.12928, 2024b. Ze Liu, Yutong Lin, Yue Cao, Ha...

    arXiv

  50. [2023]

    From novice to expert: Llm agent policy optimization via step-wise reinforcement learning.arXiv preprint arXiv:2411.03817,

    Zhirui Deng, Zhicheng Dou, Yutao Zhu, Ji-Rong Wen, Ruibin Xiong, Mang Wang, and Weipeng Chen. From novice to expert: Llm agent policy optimization via step-wise reinforcement learning.arXiv preprint arXiv:2411.03817,

    arXiv

  51. [2024]

    Beit: Bert pre-training of image transformers

    Hangbo Bao, Li Dong, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254,

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.