Pith. sign in

REVIEW 5 major objections 6 minor 47 references

Discrete JEPA: Learning Discrete Token Representations without Reconstruction

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Discrete-JEPA claims that quantizing only the global semantic representation inside a Joint-Embedding Predictive Architecture—trained with Semantic-to-Patch, Patch-to-Semantic, and Patch-to-Patch losses—yields discrete semantic tokens…

desk verdict A legitimate JEPA extension, but the headline claim about discretization is underdetermined by a confounded single-baseline comparison. read the letter →

arxiv 2506.14373 v2 pith:USDIEVTX submitted 2025-06-17 cs.CV

Junyeob Baek , Hosung Lee , Christopher Hoang , Mengye Ren , Sungjin Ahn This is my paper
classification cs.CV
keywords discretetokenizationjoint-embeddingpredictivearchitecturevectorquantizationsemanticrepresentationlearninglatentpredictionlong-horizonsymbolicreasoningworldmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Discrete-JEPA, a tokenizer that learns discrete image tokens without any pixel-reconstruction objective. It claims that vector-quantizing only the global semantic representation, while keeping patch-level representations continuous during training, gives tokens that carry the high-level scene content a world model needs for symbolic prediction. On two synthetic visual-sequence tasks, the reported result is that this discrete token space keeps color and pattern predictions accurate across 200 to 1,000 rollout steps, whereas continuous I-JEPA baselines degrade from accumulated error. A sympathetic reader would therefore take the contribution to be a demonstration that discrete semantic tokens, trained purely by latent prediction, can serve as a stable substrate for long-horizon reasoning and planning from pixels.

What carries the argument

The object that carries the argument is the vector-quantized semantic token: $L$ learnable tokens (8 or 32 per image) extracted by the context and target encoders alongside continuous patch tokens, passed through nearest-neighbor quantization into a shared codebook $C_s \in \mathbb{R}^{K_s \times D_s}$ with $K_s = 1024$ entries and $D_s = 96$. The mechanism that trains it is the unified predictive objective $L_{\mathrm{total}} = \lambda_1 L_{\mathrm{S2P}} + \lambda_2 L_{\mathrm{P2S}} + \lambda_3 L_{\mathrm{P2P}} + L_{\mathrm{VQ}}$: Semantic-to-Patch forces the discrete token to contain enough global information to reconstruct patch-level target representations, Patch-to-Semantic forces the continuous patches to contribute to the semantic token, and Patch-to-Patch keeps the original JEPA spatial-prediction ability. The discrete index space is what is claimed to prevent compounding error: each autoregressive prediction commits to a codebook entry, so small continuous drifts cannot accumulate across many rollout steps.

What would settle it

Train the same encoder with continuous semantic tokens at the same dimension and token count as the VQ output, keeping the world model, optimizer, and learning rate identical to the discrete run, and compare 200-step color accuracy on Dancing-Sprites-Pattern; if the continuous variant matches or beats the discrete one, discretization is not the cause of the stability. In the same setting, measure codebook perplexity on the test split—if a tiny fraction of the 1024 entries accounts for nearly all images, the tokens are not encoding the per-image scene state the argument requires.

Watch

Extended reading notes

Core claim

Discrete-JEPA's central claim is that the stability of long-horizon visual prediction comes from representing the scene state as a small set of discrete semantic indices rather than as continuous vectors. The architecture extends I-JEPA with $L$ learnable semantic tokens (8 per image on Dancing-Sprites-Pattern, 32 on Blinking-Ball) that the context and target encoders produce alongside continuous patch tokens; only the semantic tokens pass through a shared vector-quantization codebook. The three complementary objectives are Semantic-to-Patch prediction, Patch-to-Semantic prediction, and Patch-to-Patch prediction, combined as $L_{\mathrm{total}} = \lambda_1 L_{\mathrm{S2P}} + \lambda_2 L_{\mathrm{P2S}} + \lambda_3 L_{\mathrm{P2P}} + L_{\mathrm{VQ}}$. The paper reports that this yields perfect color accuracy across 200 rollout steps on Dancing-Sprites-Pattern while I-JEPA baselines decline, and on Blinking-Ball it maintains pattern integrity across roughly 1,000 steps with about 6x better LPIPS and 5x better MSE at step 1,000. The sustained adherence to the underlying pattern is presented as evidence of deliberate, rule-based planning in the discrete semantic token space rather than reactive next-frame prediction.

Load-bearing premise

The load-bearing premise is that a few vector-quantized semantic tokens (8 or 32 per image) trained only by latent-prediction losses carry enough scene state—color, shape, and position—for an autoregressive world model to roll the future forward accurately; if the codebook instead latches onto a shortcut specific to the synthetic training patterns, the long-horizon advantage will not transfer beyond these datasets.

Editorial extensions

If this is right

  • Color and pattern prediction on Dancing-Sprites-Pattern stays at perfect accuracy across 200 rollout steps, while both I-JEPA continuous baselines degrade over the same horizon.
  • On Blinking-Ball, Discrete-JEPA stabilizes after the first 50 steps and ends with roughly 6x better LPIPS and 5x better MSE at 1,000 steps than I-JEPA, which declines continuously.
  • The discrete semantic token can serve as the input to an autoregressive world model that predicts future token indices, making the learned representation compatible with sequence-modeling paradigms that require discrete tokens.
  • Because continuous patch tokens remain in the training loop, the method keeps spatial prediction (the P2P objective) while still outputting discrete semantic tokens, so the abstraction-precision trade-off is explicit rather than accidental.
  • The observed maintenance of color-cycle and zigzag patterns over long rollouts indicates that the model can carry an abstract symbolic rule, not just match the next frame, which is the capacity the paper argues is needed for planning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the paper's Discrete-JEPA and I-JEPA comparisons differ in token count, token dimension, world-model loss, and learning rate, a matched control with identical hyperparameters would reveal how much of the long-horizon stability is attributable to discretization itself rather than to capacity or optimization.
  • Editorial inference: the 1024-entry codebook is never analyzed for usage or per-entry semantics; decoding color, shape, and position from individual token indices would show whether the codebook organizes along the symbolic dimensions the task rewards, and would make the claimed semantic abstraction testable.
  • Editorial inference: the tokenizer is evaluated only on fixed-layout synthetic scenes; the same three objectives on real-world video would show whether long-horizon stability survives scene variation and whether a multi-granularity codebook can recover the positional precision that single-level semantic tokens give up.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Discrete-JEPA, an extension of the JEPA framework in which a small set of semantic tokens is vector-quantized while continuous patch tokens are kept for the complementary S2P, P2S, and P2P prediction objectives. At inference, an autoregressive world model predicts future discrete token indices, and either a linear prober (Dancing-Sprites-Pattern) or an image decoder (Blinking-Ball) converts predictions into symbolic properties or pixels. On two synthetic sequence datasets, Discrete-JEPA is reported to outperform continuous I-JEPA baselines over long rollout horizons, with qualitative rollout images claimed to show systematic pattern maintenance in the semantic token space. The paper's central claim is that discrete semantic tokenization eliminates the accumulating errors of continuous latent prediction and yields more stable long-horizon symbolic prediction.

Significance. If the central claim were established, the paper would make a useful contribution: it provides a concrete way to obtain discrete, semantically meaningful image tokens without pixel reconstruction, with detailed hyperparameters and dataset protocols that make the experiments reproducible. The idea of discretizing only the global semantic stream is architecturally interesting, and the empirical gap over I-JEPA is suggestive. However, the current comparison varies several factors jointly, and the qualitative interpretation is not backed by quantitative codebook analysis; the contribution is therefore promising but not yet supported at the level claimed.

major comments (5)
  1. [§5, Tables 2–3] The central claim that discretization eliminates compounding errors is not identified by the current comparison. In Table 2, Discrete-JEPA differs from I-JEPA simultaneously in token count (8 vs 64 or 1), token dimension (96 vs 768), world-model input representation (indices vs continuous vectors), and loss type (cross-entropy vs MSE), and in Table 3 additionally in encoder learning rate (1e-5 vs 1e-3) and world-model type (I2I vs R2R). Any of these factors, or their combination, could explain the stability gap in Figures 3–5 and Table 1. I request matched ablations that vary one factor at a time: a continuous-token version of Discrete-JEPA with the same S2P/P2S/P2P objectives and the same token count and dimension; a discrete version with 64 tokens and dimension 768; and I-JEPA with the same world-model loss type. Without such controls, the paper supports only the weaker claim that the full Discrete-JEPA pipeline performs well on these tasks.
  2. [Appendix B.2, Blinking-Ball protocol] The description of input preprocessing for Blinking-Ball is ambiguous and potentially circular. The text states that input images are preprocessed by resetting all ball colors to white before patchifying and feeding them to the transformer, while the task requires predicting which ball is colored and with what color. If this preprocessing removes color from the encoder observation, then the color signal is unavailable to the model and the reported color accuracy cannot be explained; if it applies only to the decoder's image-conditioning stream, the distinction must be stated and the tokenizer/encoder input must be specified precisely. Please clarify the exact data flow and show an ablation or analysis confirming that color predictions are driven by the predicted discrete tokens rather than by any residual color information in the inputs.
  3. [Appendix A.1/A.2, Fig. 3] Both datasets use fixed spatial layouts: Dancing-Sprites-Pattern has a single object at a fixed spatial position, and Blinking-Ball has four balls at fixed positions. Consequently, the position accuracy in Fig. 3 (right) is a near-constant memorized quantity, not a test of long-horizon position prediction. The discussion in §5.1.1 of a trade-off in position prediction tasks is therefore overstated. Either introduce a dataset with moving targets or explicitly state that position is a controlled constant and remove position from the list of predicted symbolic properties.
  4. [§5.1.2, Fig. 5, §6] The claim of emergent systematic and deliberate reasoning in semantic token space is supported only by qualitative rollout images. No quantitative analysis of codebook usage, codebook entropy, token frequency, or the correlation between token indices and ground-truth attributes (color, shape, position) is provided. Such statistics are needed to rule out codebook collapse or a shortcut feature and to substantiate the systematic-patterns claim.
  5. [§5.1.1, Eq. (11)] The statement that operating in a constrained discrete index space eliminates compounding errors is not established. Autoregressive prediction over discrete indices can also compound errors: a mispredicted index becomes the conditioning input for subsequent steps. The empirical stability of Discrete-JEPA in these tasks is a useful result, but the paper should either soften this claim or provide a formal argument and a diagnostic, such as a token-error propagation rate, to support it.
minor comments (6)
  1. [§4.1, Eq. (5)] In Eq. (5), \hat z_p is defined twice and there is an extra closing parenthesis; use distinct superscripts for the S2P and P2P predictions.
  2. [Appendix B.3, Table 3] The I-JEPA 'Input Tokens' entry reads 32 per image, but §B.1.2 describes 64 patch tokens for the Concat variant and 1 pooled token for the AvgPool variant; these numbers should be reconciled.
  3. [§4.3, Eq. (11)] The loss weights \lambda_1, \lambda_2, and \lambda_3 in Eq. (11) are not reported, and there is no ablation isolating the contribution of each complementary objective; this is needed to support the claim that the three objectives are complementary.
  4. [§2, §6] The abstract and Related Work position the method against reconstruction-based semantic tokenizers such as TiTok and FlexTok, but no such baseline is evaluated; the Limitations section acknowledges this, but the corresponding claims in §1 and §2 should be phrased as expectations rather than demonstrated improvements.
  5. [Fig. 1] In the Figure 1 caption, the words 'Impured' and 'Purer' appear to be typographical; please check the figure text.
  6. [§4.2–4.3] The notation for semantic targets is inconsistent: Eq. (8) uses \bar z_p^{(i)} for patch targets and Eq. (9) uses \bar z_s for the semantic target, while Eq. (4) quantizes both z_s and \bar z_s; please clarify which representation is used as the P2S target and whether gradients flow through the codebook.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method and objectives are stated up front, the evaluation is empirical, and no claimed prediction reduces by construction to a fitted input or to a self-citation.

full rationale

Discrete-JEPA does not derive a target quantity from an input by construction. The unified objective (Eq. 11) is defined from the S2P, P2S, P2P losses and a standard VQ commitment term; these are training objectives, not fitted-to-evaluation metrics. The central claims (stable long-horizon color/shape prediction, emergence of systematic token patterns) are evaluated on held-out synthetic sequence tasks with a world model and linear probes or a decoder trained separately; the metrics are not the same functions as the losses. The Blinking-Ball preprocessing (Appendix B.2) resets ball colors to white before tokenization, making the tokenizer output the sole carrier of color information; this is an intentional information bottleneck for the decoder, not a case of defining the prediction in terms of the input, because the predicted colors still must be produced by the world model from context tokens. The self-citations (Baek et al. 2025, Jiang et al. 2024) are used for background and for a dataset protocol, not as a uniqueness theorem or as the justification for the main result. The main comparison against I-JEPA is confounded by token count, dimension, loss, and learning rate, and the fixed-position synthetic tasks make position prediction trivial; these are threats to validity or experimental-design concerns, not circularity. No equation in the paper reduces to its own input, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. The ledger above lists the design choices and background assumptions the empirical claim rests on, including per-task hyperparameters and the untested assumption that the new predictive objectives are what make the tokens semantic.

free parameters (5)
  • Loss weights lambda-1, lambda-2, lambda-3
    Appear in Eq. 11 as the unified objective; values are never reported in the paper or appendix, yet they control the trade-off between the three predictive losses.
  • Number of semantic tokens L = 8 (Dancing-Sprites), 32 (Blinking-Ball)
    Hyperparameter chosen per task (Tables 2, 3); directly controls how much information the discrete bottleneck retains.
  • Codebook size K_s = 1024
    Codebook capacity for both tasks (Tables 2, 3); larger or smaller codebooks could change the abstraction-precision trade-off.
  • Masking ratio range = 40-60% (Dancing-Sprites), 50-70% (Blinking-Ball)
    Random masking ranges chosen per task; the target coverage affects what the predictor must learn.
  • SVQ quantizer configuration
    Tables 2 and 3 list the quantizer as SVQ, but the acronym and its hyperparameters are never defined in the paper.
assumptions (4)
  • standard math Vector quantization with commitment loss and EMA codebook updates follows Van Den Oord et al. (2017)
    Invoked in Section 4.2 without proof; standard background.
  • domain assumption Synthetic Spriteworld tasks with fixed layouts and hand-coded color/position patterns are valid proxies for symbolic reasoning and planning
    Section 5 and Appendix A define the evaluation; the claim that accuracy on these tasks measures systematic reasoning is assumed.
  • ad hoc to paper The complementary objectives S2P, P2S, P2P (Eqs. 8-10) are necessary and sufficient to produce semantically meaningful discrete tokens
    No ablation isolates each objective; the design is justified by intuition only.
  • ad hoc to paper A discrete index space inherently eliminates accumulated prediction error relative to continuous representations
    Stated in Section 5.1.1 as a fundamental advantage without a formal argument; depends on the fitted world model not making index errors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Discrete JEPA: Learning Discrete Token Representations without Reconstruction." pith.science (2026). https://pith.science/paper/USDIEVTX

@misc{pith2026250614373,
  author       = {Pith},
  title        = {Pith review of: Discrete JEPA: Learning Discrete Token Representations without Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/USDIEVTX}},
  note         = {Machine review of arXiv:2506.14373}
}
read the original abstract

The cornerstone of cognitive intelligence lies in extracting hidden patterns from observations and leveraging these principles to systematically predict future outcomes. However, current image tokenization methods demonstrate significant limitations in tasks requiring symbolic abstraction and logical reasoning capabilities essential for systematic inference. To address this challenge, we propose Discrete-JEPA, extending the latent predictive coding framework with semantic tokenization and novel complementary objectives to create robust tokenization for symbolic reasoning tasks. Discrete-JEPA dramatically outperforms baselines on visual symbolic prediction tasks, while striking visual evidence reveals the spontaneous emergence of deliberate systematic patterns within the learned semantic token space. Though an initial model, our approach promises a significant impact for advancing Symbolic world modeling and planning capabilities in artificial intelligence systems.

Figures

Figures reproduced from arXiv: 2506.14373 by the authors.

Figure 1
Figure 1. Discrete JEPA Overview. Existing tokenization ap￾proaches suffer from limited semantic abstraction (a) or recon￾struction bias (b). Our Discrete JEPA addresses both limitations by learning discrete semantic tokens via latent predictive coding, enabling superior symbolic reasoning capabilities. Recent advances in image tokenization (Van Den Oord et al., 2017; Esser et al., 2021; Ramesh et al., 2021; Razavi et al., 20… view at source ↗
Figure 2
Figure 2. Discrete-JEPA Architecture Overview. The context encoder f c θ takes masked inputs with learnable tokens z 0 s and generates semantic (zs) and patch (zp) representations, while the target encoder f t θ¯ processes the complete image to produce target representations z¯s and z¯p. Vector quantization (VQ) is applied only to semantic representations to create discrete tokens z discrete s . Using these discrete semantic … view at source ↗
Figure 3
Figure 3. Long-horizon prediction performance on Dancing-Sprites-Pattern dataset. Performance comparison across color (left), shape (center), and position (right) prediction tasks over 200 rollout steps. Discrete-JEPA maintains stable performance while I-JEPA variants degrade over time due to accumulated errors in continuous space. D-JEPA achieves perfect color prediction stability, highlighting the benefits of discrete seman… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Visualization of Semantic Planning on Blinking Ball. Long-horizon predictions over 1,000 timesteps. I-JEPA breaks pattern consistency around t=600 despite initial accuracy, while Discrete-JEPA maintains systematic pattern integrity throughout, demonstrating deliberate …
Figure 6
Figure 6. Figure 6: Dataset Visualization. A.1. Dancing-Sprites-Pattern Dataset The Dancing-Sprites-Pattern dataset consists of 64×64 color image sequences, each containing a single object at a fixed spatial position. Built upon the Spriteworld environment (Watters et al., 2019), each obj…
Figure 7
Figure 7. Figure 7: Additional Visualization of Semantic Planning on Blinking Ball. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 21 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Self-supervised learning from images with a joint-embedding predictive architecture

    Assran, M., Duval, Q., Misra, I., Bojanowski, P., Vincent, P., Rabbat, M., LeCun, Y., and Ballas, N. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 15619--15629, 2023

    2023

  3. [3]

    F., Amirloo, E., El-Nouby, A., Zamir, A., and Dehghan, A

    Bachmann, R., Allardice, J., Mizrahi, D., Fini, E., Kar, O. F., Amirloo, E., El-Nouby, A., Zamir, A., and Dehghan, A. Flextok: Resampling images into 1d token sequences of flexible length. arXiv preprint arXiv:2502.13967, 2025

    arXiv 2025

  4. [4]

    Dreamweaver: Learning compositional world models from pixels

    Baek, J., Wu, Y.-F., Singh, G., and Ahn, S. Dreamweaver: Learning compositional world models from pixels. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=e5mTvjXG9u

    work page 2025

  5. [5]

    Data2vec: A general framework for self-supervised learning in speech, vision and language

    Baevski, A., Hsu, W.-N., Xu, Q., Babu, A., Gu, J., and Auli, M. Data2vec: A general framework for self-supervised learning in speech, vision and language. In International conference on machine learning, pp.\ 1298--1312. PMLR, 2022

    work page 2022

  6. [6]

    Beit: Bert pre-training of image transformers

    Bao, H., Dong, L., Piao, S., and Wei, F. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021

    arXiv 2021

  7. [7]

    Vicreg: Variance-invariance-covariance regularization for self-supervised learning

    Bardes, A., Ponce, J., and LeCun, Y. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. arXiv preprint arXiv:2105.04906, 2021

    arXiv 2021

  8. [8]

    V-jepa: Latent video prediction for visual representation learning

    Bardes, A., Garrido, Q., Ponce, J., Chen, X., Rabbat, M., LeCun, Y., Assran, M., and Ballas, N. V-jepa: Latent video prediction for visual representation learning. 2023 a

    work page 2023

Show all 47 references
  1. [9]

    Mc-jepa: A joint-embedding predictive architecture for self-supervised learning of motion and content features

    Bardes, A., Ponce, J., and LeCun, Y. Mc-jepa: A joint-embedding predictive architecture for self-supervised learning of motion and content features. arXiv preprint arXiv:2307.12698, 2023 b

    2023 arXiv

  2. [10]

    Bengio, Y. et al. From system 1 deep learning to system 2 deep learning. In Neural Information Processing Systems, 2019

    2019

  3. [11]

    Unsupervised learning of visual features by contrasting cluster assignments

    Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33: 0 9912--9924, 2020

    2020

  4. [12]

    Emerging properties in self-supervised vision transformers

    Caron, M., Touvron, H., Misra, I., J \'e gou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 9650--9660, 2021

    2021

  5. [13]

    Chang, H., Zhang, H., Jiang, L., Liu, C., and Freeman, W. T. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 11315--11325, 2022

    2022

  6. [14]

    Denoising with a joint-embedding predictive architecture

    Chen, D., Hu, J., Wei, X., and Wu, E. Denoising with a joint-embedding predictive architecture. In The Thirteenth International Conference on Learning Representations, 2025 a . URL https://openreview.net/forum?id=d4njmzM7jf

    2025

  7. [15]

    Masked autoencoders are effective tokenizers for diffusion models

    Chen, H., Han, Y., Chen, F., Li, X., Wang, Y., Wang, J., Wang, Z., Liu, Z., Zou, D., and Raj, B. Masked autoencoders are effective tokenizers for diffusion models. In International Conference on Learning Representations, 2025 b

    2025

  8. [16]

    A simple framework for contrastive learning of visual representations

    Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual representations. In III, H. D. and Singh, A. (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning R...

    2020

  9. [17]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  10. [18]

    Taming transformers for high-resolution image synthesis

    Esser, P., Rombach, R., and Ommer, B. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 12873--12883, 2021

    2021

  11. [19]

    Evans, J. S. B. and Stanovich, K. E. Dual-process theories of higher cognition: Advancing the debate. Perspectives on psychological science, 8 0 (3): 0 223--241, 2013

    2013

  12. [20]

    A-jepa: Joint-embedding predictive architecture can listen

    Fei, Z., Fan, M., and Huang, J. A-jepa: Joint-embedding predictive architecture can listen. arXiv preprint arXiv:2311.15830, 2023

    2023 arXiv

  13. [21]

    R., Blundell, C., Beaudoin, P., Heess, N., Mozer, M

    Goyal, A., Didolkar, A., Ke, N. R., Blundell, C., Beaudoin, P., Heess, N., Mozer, M. C., and Bengio, Y. Neural production systems. Advances in Neural Information Processing Systems, 34: 0 25673--25687, 2021

    2021

  14. [22]

    Bootstrap your own latent-a new approach to self-supervised learning

    Grill, J.-B., Strub, F., Altch \'e , F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33: ...

    2020

  15. [23]

    Momentum contrast for unsupervised visual representation learning

    He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 9729--9738, 2020

    2020

  16. [24]

    Masked autoencoders are scalable vision learners

    He, K., Chen, X., Xie, S., Li, Y., Doll\'ar, P., and Girshick, R. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 16000--16009, June 2022

    2022

  17. [25]

    Slot state space models

    Jiang, J., Deng, F., Singh, G., Lee, M., and Ahn, S. Slot state space models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=BJv1t4XNJW

    2024

  18. [26]

    Thinking, fast and slow

    Kahneman, D. Thinking, fast and slow. Farrar, Straus and Giroux, 2011

    2011

  19. [27]

    Democratizing text-to-image masked generative models with compact text-aware one-dimensional tokens

    Kim, D., He, J., Yu, Q., Yang, C., Shen, X., Kwak, S., and Chen, L.-C. Democratizing text-to-image masked generative models with compact text-aware one-dimensional tokens. arXiv preprint arXiv:2501.07730, 2025

    2025 arXiv

  20. [28]

    A path towards autonomous machine intelligence version 0.9

    LeCun, Y. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62 0 (1): 0 1--62, 2022

    2022

  21. [29]

    Autoregressive image generation using residual quantization

    Lee, D., Kim, C., Kim, S., Cho, M., and Han, W.-S. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 11523--11532, 2022

    2022

  22. [30]

    Finite scalar quantization: Vq-vae made simple

    Mentzer, F., Minnen, D., Agustsson, E., and Tschannen, M. Finite scalar quantization: Vq-vae made simple. arXiv preprint arXiv:2309.15505, 2023

    2023 arXiv

  23. [31]

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H. V., Szafraniec, M., Khalidov, V., Fernandez, P., HAZIZA, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.-Y., Li, S.-W., Misra, I., Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Jegou, H., Maira...

    2024

  24. [32]

    Zero-shot text-to-image generation

    Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., and Sutskever, I. Zero-shot text-to-image generation. In International conference on machine learning, pp.\ 8821--8831. Pmlr, 2021

    2021

  25. [33]

    Generating diverse high-fidelity images with vq-vae-2

    Razavi, A., Van den Oord, A., and Vinyals, O. Generating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, 32, 2019

    2019

  26. [34]

    J., and Chaudhuri, S

    Sehgal, A., Grayeli, A., Sun, J. J., and Chaudhuri, S. Neurosymbolic grounding for compositional world models. arXiv preprint arXiv:2310.12690, 2023

    2023 arXiv

  27. [35]

    Joint embedding predictive architectures focus on slow features

    Sobal, V., SV, J., Jalagam, S., Carion, N., Cho, K., and LeCun, Y. Joint embedding predictive architectures focus on slow features. arXiv preprint arXiv:2211.10831, 2022

    2022 arXiv

  28. [36]

    Sq-vae: Variational bayes on discrete representation with self-annealed stochastic quantization

    Takida, Y., Shibuya, T., Liao, W., Lai, C.-H., Ohmura, J., Uesaka, T., Murata, N., Takahashi, S., Kumakura, T., and Mitsufuji, Y. Sq-vae: Variational bayes on discrete representation with self-annealed stochastic quantization. arXiv preprint arXiv:2205.07547, 2022

    2022 arXiv

  29. [37]

    Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment

    Tang, H., Key, D., and Ellis, K. Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment. Advances in Neural Information Processing Systems, 37: 0 70148--70212, 2024

    2024

  30. [38]

    and Levy, M

    Van Balen, J. and Levy, M. Pq-vae: Efficient recommendation using quantized embeddings. In RecSys (Late-Breaking Results), pp.\ 46--50, 2019

    2019

  31. [39]

    Neural discrete representation learning

    Van Den Oord, A., Vinyals, O., et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017

    2017

  32. [40]

    LARP : Tokenizing videos with a learned autoregressive generative prior

    Wang, H., Suri, S., Ren, Y., Chen, H., and Shrivastava, A. LARP : Tokenizing videos with a learned autoregressive generative prior. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=Wr3UuEx72f

    2025

  33. [41]

    Spriteworld: A flexible, configurable reinforcement learning environment

    Watters, N., Matthey, L., Borgeaud, S., Kabra, R., and Lerchner, A. Spriteworld: A flexible, configurable reinforcement learning environment. https://github.com/deepmind/spriteworld/, 2019. URL https://github.com/deepmind/spriteworld/

    2019

  34. [42]

    Towards semantic equivalence of tokenization in multimodal llm

    Wu, S., Fei, H., Li, X., Ji, J., Zhang, H., Chua, T.-S., and Yan, S. Towards semantic equivalence of tokenization in multimodal llm. arXiv preprint arXiv:2406.05127, 2024

    2024 arXiv

  35. [43]

    Temporally consistent transformers for video generation

    Yan, W., Hafner, D., James, S., and Abbeel, P. Temporally consistent transformers for video generation. In International Conference on Machine Learning, pp.\ 39062--39098. PMLR, 2023

    2023

  36. [44]

    Y., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y., Baldridge, J., and Wu, Y

    Yu, J., Li, X., Koh, J. Y., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y., Baldridge, J., and Wu, Y. Vector-quantized image modeling with improved vqgan. arXiv preprint arXiv:2110.04627, 2021

    2021 arXiv

  37. [45]

    G., Yang, M.-H., Hao, Y., Essa, I., et al

    Yu, L., Cheng, Y., Sohn, K., Lezama, J., Zhang, H., Chang, H., Hauptmann, A. G., Yang, M.-H., Hao, Y., Essa, I., et al. Magvit: Masked generative video transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 10459--10469, 2023

    2023

  38. [46]

    An image is worth 32 tokens for reconstruction and generation

    Yu, Q., Weber, M., Deng, X., Shen, X., Cremers, D., and Chen, L.-C. An image is worth 32 tokens for reconstruction and generation. Advances in Neural Information Processing Systems, 37: 0 128940--128966, 2024

    2024

  39. [47]

    ibot: Image bert pre-training with online tokenizer

    Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A., and Kong, T. ibot: Image bert pre-training with online tokenizer. International Conference on Learning Representations (ICLR), 2022

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.