Pith. sign in

REVIEW 5 major objections 6 minor 53 references

Improving Joint Embedding Predictive Architecture with Diffusion Noise

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding EDM diffusion noise to masked-token position embeddings in a Joint-Embedding Predictive Architecture improves ImageNet linear-probing accuracy by 1.5 points over a reproduced I-JEPA baseline at both 100 and 600 epochs.

desk verdict Plausible but underspecified: the +1.5% linear-probing gain over I-JEPA is not yet nailed down because the baseline schedule fix is ambiguous and all results are single seeds. read the letter →

arxiv 2507.15216 v1 pith:VQRJJNLU submitted 2025-07-21 cs.CV

Yuping Qiu , Rui Zhu , Ying-cong Chen This is my paper
classification cs.CV
keywords self-supervisedlearningjoint-embeddingpredictivearchitecturemaskedimagemodelingdiffusionnoiseEDMschedulelinearprobingNet-1Kfeatureaugmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Self-supervised vision models learn by predicting missing parts of an image, but the prediction is usually made from clean context. N-JEPA, the method in this paper, adds a dose of Gaussian diffusion noise, sampled with the EDM schedule, to the position embeddings of masked tokens before a second predictor tries to reconstruct the teacher features. The paper reports that this injection, combined with two auxiliary losses and a multi-level noise schedule, improves ImageNet linear probing to 68.3% at 100 epochs and 73.6% at 600 epochs, compared with 66.8% and 72.1% for its reproduced I-JEPA baseline. The central idea is that diffusion noise is a natural sibling of masking, and the gain acts as feature-level augmentation that needs no hand-crafted transforms. If the comparison is fair, the result shows a concrete bridge between generative-model noise and discriminative representation learning.

What carries the argument

The carrying mechanism is EDM-style noise on masked-token position embeddings: for each masked block, the model draws $n \sim \mathcal{N}(0, \sigma^2 I)$ and adds it to the block's position embedding before the second predictor sees it. Because EDM schedules noise by level $\sigma$ rather than by timestep $t$, the ViT needs no timestep embedding, so the architecture stays the same as I-JEPA. A multi-level noise schedule samples a fresh noise vector per masked block, acting as feature-level augmentation. Two auxiliary losses, $L_{\mathrm{N-T}}$ (predicting teacher representations from noisy positions) and $L_{\mathrm{C-N}}$ (denoising the noisy predictions toward the context predictor's output), are added to the context-teacher loss $L_{\mathrm{C-T}}$ with small weights $\lambda_1 = \lambda_2 = 0.1$; the total is $L_{\mathrm{total}} = L_{\mathrm{C-T}} + \lambda_1 L_{\mathrm{N-T}} + \lambda_2 L_{\mathrm{C-N}}$.

What would settle it

Retrain the reproduced I-JEPA baseline and N-JEPA from shared seeds with the corrected schedule applied symmetrically (same batch size, same mask distributions, same ipescale = 1.25) and check whether N-JEPA still beats the baseline by 1.5 points on ImageNet-1K linear probing at 100 and 600 epochs; if the margin shrinks or reverses, the central claim is unsupported.

Watch

Extended reading notes

Core claim

The paper argues that within a joint-embedding predictive architecture, injecting diffusion noise into the position embeddings of masked blocks makes the learned representations more linear-separable and more transferable. In N-JEPA, each of the four teacher mask blocks receives its own EDM-sampled Gaussian noise added to its position embedding, and two narrow predictors map the visible context to the teacher representations: one with clean positions and one with noised positions. Training uses three terms: a context-teacher prediction loss, a noise-teacher prediction loss, and a context-noise denoising loss. On ImageNet-1K linear probing, the method reaches 68.3% at 100 epochs and 73.6% at 600 epochs, which is +1.5% over the reproduced I-JEPA baseline in both schedules, and it also improves 1% and 10% label semi-supervised accuracy by 2.5 and 3.1 points using the last four layers.

Load-bearing premise

The +1.5% gain over I-JEPA rests on the assumption that the reproduced I-JEPA baseline and N-JEPA were trained with exactly the same corrected hyper-parameter schedule; the paper fixes a 25% scheduler overrun by setting ipescale = 1.25 but does not state explicitly that both models received the same fix.

Editorial extensions

If this is right

  • At the same corrected schedule, N-JEPA beats the reproduced I-JEPA baseline by 1.5 points in ImageNet-1K linear probing at both 100 and 600 epochs (68.3 vs 66.8 and 73.6 vs 72.1).
  • Low-shot transfer improves with the frozen encoder: using the last four layers gives +2.5 points on 1% of ImageNet labels and +3.1 points on 10%.
  • The auxiliary losses help only at small weights; setting both $\lambda_1$ and $\lambda_2$ to 0.1 gives +1.1 points at 100 epochs and +1.3 points at 600, while weight 1.0 yields only +0.4 points.
  • A multi-level noise schedule beats a single-level one by 0.4 points at 100 epochs and 0.2 points at 600, supporting the feature-augmentation interpretation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the reported gains are on ViT-B/16 only; the natural scaled test is whether the 1.5-point improvement persists with ViT-L or ViT-H and in dense prediction tasks such as detection and segmentation.
  • Beyond the paper: because noise is added only to masked-token position embeddings, ablating noise on visible-token positions or on patch features would separate a positional-uncertainty effect from a general denoising-auxiliary effect.
  • Beyond the paper: the multi-level noise schedule is effectively random perturbation of the positional signal; making noise levels learnable or conditioning the predictor on the injected level could test whether the model treats the noise level as a usable signal.
  • Beyond the paper: if diffusion noise and masking are continuous cousins, a schedule interpolating between pure masking and pure Gaussian noise would provide a direct way to map the spectrum of corruption types for representation learning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes N-JEPA, a modification of I-JEPA that adds EDM-style Gaussian noise to the position embeddings of masked tokens before prediction, uses two predictors (one with and one without noise), and trains with three losses: context-teacher, noise-teacher, and a denoising consistency loss. A multi-level noise schedule assigns a different draw of noise to each masked block. The authors pre-train ViT-B/16 on ImageNet-1K for 100 or 600 epochs and evaluate with linear probing on full and low-shot labels. They report consistent gains over a reproduced I-JEPA baseline (+1.5% top-1 at both 100 and 600 epochs, and +2.5/+3.1% on 1%/10% labels) and attribute them to the multi-level noise schedule and auxiliary losses.

Significance. If the reported gains hold, the paper makes a modest but useful contribution: a simple, low-cost way to inject diffusion-style noise into JEPA without changing the ViT backbone or adding timestep embeddings. The design is clean and the results are internally consistent across the 100/600-epoch and low-shot tables. The paper is also honest about its scope (ViT-B only, no generation tasks). However, the manuscript does not currently provide code or seeds, and the headline comparison depends on an incompletely specified scheduler correction; the contribution is therefore plausible rather than fully established. The idea of interpreting multi-level noise as feature augmentation is a strength.

major comments (5)
  1. [Sec. 4.1 and Table 4] The central comparison against I-JEPA is not yet fully supported. The text explains that a scheduler bug made all hyper-parameter schedules run 25% beyond the actual training schedule and that setting ipescale = 1.25 truncates the schedulers, but it does not state whether the reproduced I-JEPA baseline in Table 4 was trained with this correction, nor whether the 100-epoch runs used it. If the baseline used the uncorrected schedule while N-JEPA used the truncated one, part or all of the claimed +1.5% could be a scheduling artifact. Please specify the exact scheduler configuration used for each run in Tables 1-6.
  2. [Sec. 4.5, Table 2] The final hyper-parameters λ1 = 0.1 and λ2 = 0.1 are chosen by maximizing the same ImageNet linear-probing accuracy that is later reported as the headline result in Table 4. No held-out validation set or selection protocol is described, so the reported gain may partly reflect selection on the evaluation target. Please either select hyper-parameters on a validation split and report test results, or show that the gain is robust across a range of weights.
  3. [Tables 4 and 6] All reported numbers come from single runs; no seeds, standard deviations, or significance tests are provided. Since the headline gain is only 1.5 percentage points and the 100-epoch setting uses batch size 128 while the 600-epoch setting uses batch size 1024, the paper should either report multiple seeds with variance or justify why seed-to-seed variation is negligible.
  4. [Sec. 4.2 vs. Appendix A.2] There is an internal contradiction in the predictor architecture: Sec. 4.2 states the predictor has depth 6 and embedding dimension 384, while the Appendix A.2 config in Figure 4 lists predictor depth 12 and predictor attention heads 16. Since the predictor is a core component of the method, this ambiguity must be resolved before the results can be reproduced.
  5. [Sec. 2.1 and Sec. 3.2] FlexPredict is described as the closest method that injects stochastic noise into position embeddings for masked image modeling, but no FlexPredict baseline is evaluated. Without such a comparison, the specific contribution of EDM noise in position embeddings is not isolated from the contributions of the two-predictor design, the auxiliary losses, and the multi-level noise schedule; at minimum, the paper should discuss or provide this comparison.
minor comments (6)
  1. [Introduction and Sec. 2.1] There is a typo in the Introduction: 'Joint-Embedding Predictive Architecture (JEPA])' should read 'JEPA', and 'MDTet al. [15]' should read 'MDT et al. [15]'.
  2. [Eqs. (7)-(9)] The notation D(·,·) is used for the smooth-L1 loss without a formal definition; please define it once before using it.
  3. [Sec. 4.4] The 100-epoch and 600-epoch runs use different batch sizes (128 vs 1024); please state explicitly whether the reproduced I-JEPA baseline used the same batch size as N-JEPA at each epoch count.
  4. [Table 4] The asterisk denotes reproduced results, but the reproduction configuration (code base, mask sampling, augmentation) is not given; please provide the exact setup or a pointer to code.
  5. [Algorithm 1] The pseudo-code updates 'θ, ϕsN, ϕsc' but the text defines the predictors as gϕc and gϕn; the notation should be aligned.
  6. [Sec. 2.1] The citation '[14]' is attributed to 'Adam et al.' in the text, but the listed authors are Letian Fu et al.; please correct the citation.

Circularity Check

1 steps flagged · score 4.0 of 10

The central +1.5% linear-probing gain is partially fitted to the same ImageNet linear-probing metric used for model selection, though low-shot results provide independent support.

  1. fitted input called prediction [Section 4.5, Tables 2-4 (especially Table 2 and Table 4)]
    "Throughout the linear evaluation, as shown in table 2, we conduct an exploration of various weight settings and table 4 provides a comprehensive report on the Top-1 accuracy. ... The weights of loss hyper-parameters. We see a large performance improvement with lower weights of N-T loss and N-C loss. (+1.1%, +1.3% for 100 and 600 epochs VS. baseline.) ... Our method leads to consistent linear probing improvement compared with other methods, resulting in +1.5% improvement on both 100 and 600 epochs settings compared with I-JEPA."

    The loss weights (λ1=0.1, λ2=0.1) and the multi-level noise option were selected by comparing ImageNet linear-probing top-1 accuracy in Tables 2 and 3, and the final headline in Table 4 reports the linear-probing top-1 accuracy of exactly that selected configuration. Thus the claimed +1.5% over I-JEPA is the value of the selection objective after optimizing over these design choices, not an independent held-out prediction. The contribution is therefore partially fitted to the benchmark on which it is confirmed. The 1% and 10% low-shot evaluations in Table 6b were not used for selection and provide partial independent grounding, so the circularity is partial rather than complete.

full rationale

The paper's mechanism is not self-referential: N-JEPA adds EDM noise to masked position embeddings, defines three losses, and compares against an external I-JEPA reproduction, so the core architecture is not derived from its own output. No self-citation chain or imported uniqueness theorem is present. The one genuine circularity concern is empirical selection: the authors tuned loss weights and the noise schedule on ImageNet linear probing, then reported ImageNet linear probing as the central evidence of improvement. That makes the headline +1.5% partly a summary of the tuning sweep rather than an independent confirmation. The schedule-correction ambiguity (ipescale=1.25) and the absence of error bars are correctness and reporting risks, not circularity, because they concern whether the baseline was trained comparably, not whether the result is defined by its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on standard diffusion/EDM background (borrowed as prior math), the domain assumption that noise on position embeddings is a useful augmentation, and SSL evaluation assumptions. It also depends on three groups of chosen hyperparameters: the loss weights, the EDM noise parameters, and the scheduler truncation factor. No invented entities are introduced.

free parameters (3)
  • loss weights lambda_1 (noise-teacher), lambda_2 (denoise) = 0.1, 0.1 (best; equal-weight baseline 1.0, 1.0 also tested)
    Selected in Sec 4.5 Table 2 by maximizing the same ImageNet linear-probing metric reported as the final result; no held-out validation or error bars.
  • EDM noise distribution hyperparameters (Pmean, Pstd, sigma_data) = -1.2, 1.2, 0.5
    Taken from EDM and applied to position embeddings without ablation; Algorithm 1 actually samples n ~ N(0, sigma_data^2 I), so the sigma level is effectively fixed rather than following a multi-level EDM schedule.
  • scheduler truncation factor ipescale = 1.25
    Introduced in Sec 4.1 after observing that 80/550-epoch checkpoints outscored 100/600-epoch ones; the correction is essential to the reported 600-epoch comparison but its application to the I-JEPA baseline is not explicitly documented.
assumptions (4)
  • standard math Score-based diffusion and EDM formalisms (P_sigma(x)=P_data * N(0,sigma^2 I), denoiser D(x;sigma)) are valid and can be applied outside generative training.
    Sec 3.1 uses EDM/DDPM equations and score-SDE relations without proof; if this background is misapplied, the method's motivation fails.
  • domain assumption Adding Gaussian noise to masked-token position embeddings acts as beneficial feature augmentation for representation learning.
    Core idea of Sec 1 and Sec 3.2; only supported by single-run linear-probing comparisons in Table 3, not by theory or cross-task evidence.
  • domain assumption Teacher EMA with detached targets provides stable supervision; the three-term loss does not cause representation collapse.
    Algorithm 1 detaches z_y and updates teacher by EMA; this standard SSL assumption is inherited from I-JEPA and not reinvestigated.
  • domain assumption Linear probing and low-shot ImageNet-1K classification are sufficient proxies for representation quality.
    All conclusions are drawn from IN-1K linear probing (Sec 4.5) and 1%/10% labels (Table 6b); no detection, segmentation, transfer, or generation benchmarks are included.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Joint Embedding Predictive Architecture with Diffusion Noise." pith.science (2026). https://pith.science/paper/VQRJJNLU

@misc{pith2026250715216,
  author       = {Pith},
  title        = {Pith review of: Improving Joint Embedding Predictive Architecture with Diffusion Noise},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VQRJJNLU}},
  note         = {Machine review of arXiv:2507.15216}
}
read the original abstract

Self-supervised learning has become an incredibly successful method for feature learning, widely applied to many downstream tasks. It has proven especially effective for discriminative tasks, surpassing the trending generative models. However, generative models perform better in image generation and detail enhancement. Thus, it is natural for us to find a connection between SSL and generative models to further enhance the representation capacity of SSL. As generative models can create new samples by approximating the data distribution, such modeling should also lead to a semantic understanding of the raw visual data, which is necessary for recognition tasks. This enlightens us to combine the core principle of the diffusion model: diffusion noise, with SSL to learn a competitive recognition model. Specifically, diffusion noise can be viewed as a particular state of mask that reveals a close relationship between masked image modeling (MIM) and diffusion models. In this paper, we propose N-JEPA (Noise-based JEPA) to incorporate diffusion noise into MIM by the position embedding of masked tokens. The multi-level noise schedule is a series of feature augmentations to further enhance the robustness of our model. We perform a comprehensive study to confirm its effectiveness in the classification of downstream tasks. Codes will be released soon in public.

Figures

Figures reproduced from arXiv: 2507.15216 by the authors.

Figure 1
Figure 1. Main types of self-supervised learning. (a) Contrastive Learning: The augmented two views are fed into the student and teacher encoder, and the contrastive loss aims at pulling two feature embeddings closer. (b) MIM: The reconstructive loss aims at recovering pixel or feature-level tokens with the input image as the label. (c) N-JEPA: Our goal is to predict the teacher feature from the noised feature in the encoder … view at source ↗
Figure 2
Figure 2. The overview of our N-JEPA. The image X will be converted into a sequence of N non-overlapping patches and fed into the teacher encoder, and we feed only visible patches to the student encoder. We aim to predict the representations of various masked blocks shown in different colors (red, yellow, blue). Whether adding mutli-level noise schedule is the difference between two predictors, other settings are the same. Th… view at source ↗
Figure 4
Figure 4. Pretraining setting for downstream tasks (ViT-B). All mod￾els trained for 600 epochs. Algorithm 1 N-JEPA pseudo-code 1: Input: num iterations K, image dist D, hyper-parameter σdata, 2: encoder fθ, target-encoder fθ¯, predictor-context gϕc , predictor-noise gϕn , scalar q 3: masked position embeddings - ψsc , ψsN for predictor 1 and predictor 2. Num of mask blocks L 4: Initialize: ¯θ = θ 5: for i = 1, 2, ..., K do 6:… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 27 canonical work pages

  1. [1]

    Self-supervised learning from images with a joint-embedding predictive architecture, 2023

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture, 2023

    2023

  2. [2]

    A cookbook of self-supervised learning, 2023

    Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yann LeCun, and Micah Goldblum. A cookbook of self-supervised learning, 2023

    work page 2023

  3. [3]

    All are worth words: a vit backbone for score-based diffusion models

    Fan Bao, Chongxuan Li, Yue Cao, and Jun Zhu. All are worth words: a vit backbone for score-based diffusion models. arXiv preprint arXiv:2209.12152, 2022

    arXiv 2022

  4. [4]

    Beit: Bert pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021

    arXiv 2021

  5. [5]

    Stochastic positional embeddings improve masked image modeling, 2024

    Amir Bar, Florian Bordes, Assaf Shocher, Mahmoud Assran, Pascal Vincent, Nicolas Ballas, Trevor Darrell, Amir Globerson, and Yann LeCun. Stochastic positional embeddings improve masked image modeling, 2024

    work page 2024

  6. [6]

    Unsuper- vised learning of visual features by contrasting cluster assignments, 2021

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsuper- vised learning of visual features by contrasting cluster assignments, 2021

    work page 2021

  7. [7]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision , pages 9650–9660, 2021

    2021

  8. [8]

    Big self- supervised models are strong semi-supervised learners

    Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self- supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33:22243–22255, 2020

    work page 2020

Show all 53 references
  1. [9]

    Improved baselines with momentum contrastive learning, 2020

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning, 2020

    2020

  2. [10]

    Context autoencoder for self-supervised representation learning, 2023

    Xiaokang Chen, Mingyu Ding, Xiaodi Wang, Ying Xin, Shentong Mo, Yunhao Wang, Shumin Han, Ping Luo, Gang Zeng, and Jingdong Wang. Context autoencoder for self-supervised representation learning, 2023

    2023

  3. [11]

    Deconstructing denoising diffusion models for self-supervised learning, 2024

    Xinlei Chen, Zhuang Liu, Saining Xie, and Kaiming He. Deconstructing denoising diffusion models for self-supervised learning, 2024

    2024

  4. [12]

    Bert: Pre-training of deep bidirec- tional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirec- tional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  5. [13]

    Hospedales

    Linus Ericsson, Henry Gouk, Chen Change Loy, and Timothy M. Hospedales. Self-supervised representa- tion learning: Introduction, advances, and challenges. IEEE Signal Processing Magazine, 39(3):42–62, 2022

    2022

  6. [14]

    Rethinking patch dependence for masked autoencoders

    Letian Fu, Long Lian, Renhao Wang, Baifeng Shi, Xudong Wang, Adam Yala, Trevor Darrell, Alexei A Efros, and Ken Goldberg. Rethinking patch dependence for masked autoencoders. arXiv preprint arXiv:2401.14391, 2024

    2024 arXiv

  7. [15]

    Masked diffusion transformer is a strong image synthesizer

    Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer. arXiv preprint arXiv:2303.14389, 2023

    2023 arXiv

  8. [16]

    Learning and leveraging world models in visual representation learning, 2024

    Quentin Garrido, Mahmoud Assran, Nicolas Ballas, Adrien Bardes, Laurent Najman, and Yann LeCun. Learning and leveraging world models in visual representation learning, 2024

    2024

  9. [17]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural i...

    2020

  10. [18]

    Momentum contrast for unsupervised visual representation learning, 2020

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning, 2020

    2020

  11. [19]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022. 10

    2022

  12. [20]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  13. [21]

    What to hide from your students: Attention-guided masked im- age modeling

    Ioannis Kakogeorgiou, Spyros Gidaris, Bill Psomas, Yannis Avrithis, Andrei Bursuc, Konstantinos Karantzalos, and Nikos Komodakis. What to hide from your students: Attention-guided masked im- age modeling. In European Conference on Computer Vision, pages 300–318. Springer, 2022

    2022

  14. [22]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems , 35:26565–26577, 2022

    2022

  15. [23]

    Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation

    Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and Il-Chul Moon. Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation. arXiv preprint arXiv:2106.05527, 2021

    2021 arXiv

  16. [24]

    Exploring the role of mean teachers in self-supervised masked auto-encoders, 2022

    Youngwan Lee, Jeffrey Willette, Jonghee Kim, Juho Lee, and Sung Ju Hwang. Exploring the role of mean teachers in self-supervised masked auto-encoders, 2022

    2022

  17. [25]

    Dreamteacher: Pretraining image backbones with deep generative models

    Daiqing Li, Huan Ling, Amlan Kar, David Acuna, Seung Wook Kim, Karsten Kreis, Antonio Torralba, and Sanja Fidler. Dreamteacher: Pretraining image backbones with deep generative models. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 16698–16708, 2023

    2023

  18. [26]

    Metaug: Contrastive learning via meta feature augmentation, 2023

    Jiangmeng Li, Wenwen Qiang, Changwen Zheng, Bing Su, and Hui Xiong. Metaug: Contrastive learning via meta feature augmentation, 2023

    2023

  19. [27]

    Training stacked denoising autoencoders for representation learning, 2021

    Jason Liang and Keith Kelly. Training stacked denoising autoencoders for representation learning, 2021

    2021

  20. [28]

    Self-supervised learning: Generative or contrastive

    Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. Self-supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering , page 1–1, 2021

    2021

  21. [29]

    Latent consistency models: Synthesizing high-resolution images with few-step inference

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023

    2023 arXiv

  22. [30]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14297–14306, 2023

    2023

  23. [31]

    From points to functions: Infinite- dimensional representations in diffusion models, 2022

    Sarthak Mittal, Guillaume Lajoie, Stefan Bauer, and Arash Mehrjou. From points to functions: Infinite- dimensional representations in diffusion models, 2022

    2022

  24. [32]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models, 2022

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models, 2022

    2022

  25. [33]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning , pages 8162–8171. PMLR, 2021

    2021

  26. [34]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4195–4205, 2023

    2023

  27. [35]

    Hierarchical text-conditional image generation with clip latents, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents, 2022

    2022

  28. [36]

    High-resolution image synthesis with latent diffusion models, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models, 2022

    2022

  29. [37]

    Numerical treatment of stochastic differential equations

    W Rüemelin. Numerical treatment of stochastic differential equations. SIAM Journal on Numerical Analysis, 19(3):604–613, 1982

    1982

  30. [38]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512, 2022

    2022 arXiv

  31. [39]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  32. [40]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems , 32, 2019. 11

    2019

  33. [41]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  34. [42]

    Maximum likelihood training of score-based diffusion models

    Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems , 34:1415–1428, 2021

    2021

  35. [43]

    Siamese image modeling for self-supervised vision representation learning, 2022

    Chenxin Tao, Xizhou Zhu, Weijie Su, Gao Huang, Bin Li, Jie Zhou, Yu Qiao, Xiaogang Wang, and Jifeng Dai. Siamese image modeling for self-supervised vision representation learning, 2022

    2022

  36. [44]

    Vikas Verma, Minh-Thang Luong, Kenji Kawaguchi, Hieu Pham, and Quoc V . Le. Towards domain- agnostic contrastive learning, 2021

    2021

  37. [45]

    Masked feature prediction for self-supervised visual pre-training

    Chen Wei, Haoqi Fan, Saining Xie, Chao-Yuan Wu, Alan Yuille, and Christoph Feichtenhofer. Masked feature prediction for self-supervised visual pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14668–14678, 2022

    2022

  38. [46]

    Diffusion models as masked autoencoders

    Chen Wei, Karttikeya Mangalam, Po-Yao Huang, Yanghao Li, Haoqi Fan, Hu Xu, Huiyu Wang, Cihang Xie, Alan Yuille, and Christoph Feichtenhofer. Diffusion models as masked autoencoders. arXiv preprint arXiv:2304.03283, 2023

    2023 arXiv

  39. [47]

    Hallucination improves the performance of unsupervised visual representation learning, 2023

    Jing Wu, Jennifer Hobbs, and Naira Hovakimyan. Hallucination improves the performance of unsupervised visual representation learning, 2023

    2023

  40. [48]

    Denoising diffusion autoencoders are unified self-supervised learners, 2023

    Weilai Xiang, Hongyu Yang, Di Huang, and Yunhong Wang. Denoising diffusion autoencoders are unified self-supervised learners, 2023

    2023

  41. [49]

    Simmim: A simple framework for masked image modeling

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9653–9663, 2022

    2022

  42. [50]

    Your vit is secretly a hybrid discriminative-generative diffusion model

    Xiulong Yang, Sheng-Min Shih, Yinlin Fu, Xiaoting Zhao, and Shihao Ji. Your vit is secretly a hybrid discriminative-generative diffusion model. arXiv preprint arXiv:2208.07791, 2022

    2022 arXiv

  43. [51]

    Mobilediffusion: Subsecond text-to-image generation on mobile devices

    Yang Zhao, Yanwu Xu, Zhisheng Xiao, and Tingbo Hou. Mobilediffusion: Subsecond text-to-image generation on mobile devices. arXiv preprint arXiv:2311.16567, 2023

    2023 arXiv

  44. [52]

    Fast training of diffusion models with masked transformers

    Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers. arXiv preprint arXiv:2306.09305, 2023

    2023 arXiv

  45. [53]

    ibot: Image bert pre-training with online tokenizer

    Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832, 2021. 12 A Appendix A.1 Discussion In our paper, we do not use larger ViT models such as ViT-L/16 and ViT-...

    2021 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.