Pith. sign in

REVIEW 4 major objections 6 minor 73 references

Compositional Video Synthesis by Temporal Object-Centric Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single object-centric slot representation, learned without supervision, can both segment real videos and drive a frozen diffusion model to synthesize them pixel-by-pixel with stable identities across time.

desk verdict Solid first step at object-centric video generation with diffusion, but temporal identity stability is unmeasured and the generation baselines are image-only. read the letter →

arxiv 2507.20855 v1 pith:NKXMZ4OQ submitted 2025-07-28 cs.CV

Adil Kaan Akan , Yucel Yemez This is my paper
classification cs.CV
keywords object-centriclearningvideogenerationcompositionaleditingslotattentiondiffusionmodelsunsupervisedobjectsegmentationtemporalconsistencyself-supervised
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the same object-centric slots used for unsupervised video segmentation can also drive a pretrained image diffusion model to synthesize full videos with stable object identities, unifying perception and generation in one self-supervised framework. The authors extend their image-level method, SlotAdapt, into time by learning pose-invariant slots per frame with shared initialization, aggregating them across a short temporal window with a transformer, and conditioning a frozen Stable Diffusion decoder through adapter cross-attention layers. They report state-of-the-art foreground clustering (FG-ARI) on YTVIS19 and DAVIS17 while setting new benchmarks on PSNR, SSIM, LPIPS, FID, and FVD against image-based object-centric generative baselines, and they show the slots support compositional edits such as deleting or replacing an object. The motivation is that explicit object-level structure gives video models compositional control that holistic text-to-video models lack.

What carries the argument

The load-bearing mechanism is Invariant Slot Attention (ISA) applied per frame with shared slot initialization, followed by a transformer-based temporal aggregator that fuses slots across a five-frame window; the resulting temporally enriched slots condition a frozen Stable Diffusion decoder through adapter cross-attention layers, while DINO-pooled register tokens pass through a separate temporal aggregator and enter via the native text cross-attention to supply pose and scene context. This split keeps object identity in pose-invariant slots and spatial or pose information in register tokens, and the paper demonstrates with ablations that without register tokens objects appear in wrong positions, scales, or orientations and temporal coherence degrades sharply.

What would settle it

Take a trained model and a video in which two similar objects cross or partly occlude each other; run sliding-window inference and compare the Hungarian-matched slot sequence with ground-truth object tracks. If slots swap identities on such clips while FVD stays low, then the reported temporal coherence does not require identity stability, whereas if swapping occurs and FVD jumps, identity alignment is the load-bearing factor. Alternatively, replacing Hungarian matching with fixed slot ordering and measuring the FVD change would show whether identity alignment or slot content drives the temporal metric.

Watch

Extended reading notes

Core claim

The central discovery is that pose-invariant object slots, augmented with temporally aggregated context and global register tokens that carry spatial pose, can be injected into a frozen pretrained diffusion model via lightweight adapters to reconstruct and edit real-world video frames at pixel level while preserving object identities over time. The paper presents this as the first self-supervised object-centric video generation framework on real-world videos with compositional editing, and it argues that segmentation and generation mutually benefit: clustering quality approaches that of dedicated unsupervised methods, while generation metrics exceed per-frame-trained object-centric image diffusion baselines across every reported measure.

Load-bearing premise

The temporal-coherence claim rests on the assumption that per-frame slots, after shared initialization and temporal aggregation, keep each object's identity stable enough that Hungarian matching at inference never swaps identities; the paper reports no identity-switch or temporal-boundary metric, and since decoding is per-frame with a frozen diffusion model, a slot that drifts mid-video cannot be corrected downstream.

Editorial extensions

If this is right

  • If the claim holds, object-centric video generation no longer needs optical flow, depth, or architectural slot-transition constraints; raw video alone suffices for learning temporally consistent slots.
  • The same slot set can be edited compositionally by modifying slot vectors (removing, inserting, or replacing an object), and the frozen diffusion decoder will re-render coherent frames around that edit.
  • Segmentation and generation can be unified in one architecture without a task-specific decoder, so one trained model serves both unsupervised video object segmentation and video synthesis.
  • The reported gains across all five generation metrics suggest that object-level conditioning does not trade fidelity for structure; it improves both simultaneously.
  • Performance on FG-ARI being state of the art while mIoU slightly trails a dedicated decoder method indicates the encoder masks are competitive enough for downstream generative use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test would track slot-identity switches across long sequences using ground-truth object trajectories; the paper reports no such metric, so identity stability under crossing or occluding objects remains an open question rather than a demonstrated property.
  • Because decoding is per-frame through a frozen image diffusion model, the architecture inherits image priors and a finite temporal window; replacing the decoder with a video diffusion model, as the conclusion suggests, would test whether longer-range motion coherence improves beyond the current five-frame aggregation.
  • The compositional evaluation mixes slots between videos; a complementary stress test would edit a single slot's semantics and measure whether the rest of the scene and motion remain unchanged.
  • The large FVD drop when register tokens are removed suggests temporal coherence is carried substantially by global context vectors rather than slot identity alone, so perturbing those tokens on long clips would reveal how much of the reported consistency depends on pose and context information.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a self-supervised object-centric framework for video synthesis and editing. It extends SlotAdapt from images to video by combining a frozen DINOv2 backbone, per-frame Invariant Slot Attention with shared initialization, a transformer-based temporal aggregator over slots and register tokens, and a frozen Stable Diffusion decoder conditioned through adapter cross-attention layers. Training uses a 1-frame diffusion objective with an L-frame temporal context; inference decodes the central frame of sliding windows and aligns slot identities by Hungarian matching. Experiments report unsupervised segmentation (mIoU, FG-ARI) and generation metrics (PSNR, SSIM, LPIPS, FID, FVD) on a 300-video subset of YTVIS19 training data and on DAVIS17, plus ablations and qualitative compositional editing results.

Significance. The paper addresses a real gap: most temporal object-centric models only segment or track, and few couple unsupervised object discovery with pixel-level diffusion synthesis on real-world video. The architecture is a clean combination of published components, and the ablations (Table 1 and Appendix Table 5) provide useful evidence for the roles of ISA, register tokens, and temporal aggregation. The supplementary material includes extensive qualitative multi-frame comparisons. If the temporal-consistency claims were directly quantified, this would be a solid contribution. Currently, however, the evidence does not support the headline claims: temporal identity stability is asserted but unmeasured, the generation baselines are image-based, and the video metrics are computed on reconstructed training videos from a 300-video subset. The contribution is promising but needs substantial evaluation strengthening.

major comments (4)
  1. [Sec. 4.2 / Inference] The paper's central claim of 'superior temporal coherence' and 'consistent object identities across frames' (Abstract; Sec. 1) rests on slot identity stability that is never directly measured. The decoder is a frozen per-frame Stable Diffusion model (Sec. 4.2), the default training objective Eq. (10) denoises only one randomly selected frame, and there is no cross-frame loss or temporal decoder; temporal consistency must therefore be produced entirely by shared-init ISA, the transformer aggregator (Eqs. 7-9), and Hungarian matching at inference. The reported FVD (Table 3) is a distribution-level metric computed on reconstructed videos and cannot detect slot drift or identity swaps. I recommend adding explicit identity-preservation metrics, e.g., the rate of slot identity switches across frames, per-slot mask IoU over time, or propagation accuracy of edited objects, and a user study of edited videos. The authors' own future-work sentence in Sec. 6, replacing the image-based decoder with a video diffusion model, acknowledges this limitation.
  2. [Sec. 5.1 / Sec. 5.4, Tables 3-4] All generation metrics are computed on a 300-video subset of the YTVIS19 training set, and the paper says this is because YTVIS19 'lacks an official validation or test set with provided ground-truth masks.' This is a load-bearing evaluation choice: FID and FVD on training reconstructions measure fidelity to seen data, not generalization, so the phrase 'new benchmarks in video generation quality' (Sec. 6) overstates what the numbers show. The claim about missing YTVIS validation annotations also appears inconsistent with the dataset documentation, which includes a validation split with mask annotations. Please either evaluate on the held-out validation split or clearly relabel the results as reconstruction quality on a training subset, and state the limitation explicitly. In addition, the FVD computation (number of clips, feature backbone, sequence length) is not specified.
  3. [Sec. 5.3, Table 3] The generation baselines (LSD, SlotDiffusion, SlotAdapt) are image-generation models trained on flattened video frames as independent images (Appendix A.4). Comparing against them cannot establish 'video generation quality' or 'temporal consistency' because none of the baselines has a temporal model. The FVD improvements in Table 3 could be entirely driven by better per-frame reconstruction. Please add at least one video-capable comparison (e.g., a per-frame SlotAdapt variant with temporal slot alignment, or a video diffusion baseline run under the same compute budget), or alternatively restrict the claims to image-quality reconstruction and support temporal coherence with the identity-stability metrics suggested in Major 1.
  4. [Table 2 and footnote 2] The segmentation comparison that underpins 'state-of-the-art FG-ARI' mixes resolutions: the full SOLV numbers were reported at 336x504, while all numbers in the paper are at 224x224, and the footnote acknowledges this. The same-resolution SOLV-E and SOLV-E+M variants are below our method, but the headline comparison against full SOLV is not apples-to-apples. The absolute FG-ARI margins over SOLV (22.40 vs. 21.55 on YTVIS; 21.60 vs. 20.98 on DAVIS17) are also small, with no confidence intervals or significance tests. Moreover, our mIoU is lower than SOLV's on both datasets (40.57 vs. 42.01; 34.93 vs. 36.62), so the segmentation story should be framed as a trade-off, not uniform improvement. Please run the full SOLV model at 224x224, report variance, or soften the 'state-of-the-art' claim to 'competitive' in the affected places.
minor comments (6)
  1. [Table 2] The LSD row reports identical YTVIS and DAVIS mIoU values (29.55/29.55), which is likely a typographical error and should be corrected.
  2. [Eq. (7)] The notation ~S_{1:T} uses T while the video segment length is L throughout; use ~S_{1:L} for consistency with Eq. (8).
  3. [Sec. 4.2, Eq. (10)] The objective should specify the expectation over the random frame index, diffusion timestep, and noise; as written it reads as a single-sample loss.
  4. [Sec. 4.2 (Inference)] The Hungarian matching procedure is described only as 'based on slot similarity'; please specify the similarity measure, the matching scope (within or across windows), and how occlusions or slot dropout are handled, since this is central to the temporal-identity claim.
  5. [Sec. 5.4] mIoU is not defined precisely for frames with no foreground objects, and it is unclear whether the reported numbers are computed at 224x224 or after upsampling to original resolution (Appendix A.1).
  6. [Appendix C] The supplementary figures (Figs. 6-14, 15-25) are not referenced from the main text; please add explicit pointers, and correct the typo 'slot-basedtemporal' in Appendix C.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the core derivation is self-contained and externally benchmarked; SlotAdapt and SOLV citations are legitimate prior work rather than load-bearing self-referential reductions.

full rationale

The paper's derivation chain is a composition of published components: frozen DINOv2 features, per-frame Invariant Slot Attention, a Transformer temporal aggregator, and adapter-conditioned frozen Stable Diffusion decoding. The training objective (Eq. 10) is a standard diffusion noise-prediction loss, and the reported segmentation metrics come from the ISA attention masks (Eqs. 5, 11-17) evaluated against ground-truth masks, while generation metrics compare reconstructed frames to ground-truth frames. No equation defines its output in terms of the quantity it is claimed to predict, and no fitted parameter is renamed as a prediction. The SlotAdapt self-citation is architecturally load-bearing in the sense that the adapters are taken from that prior work, but SlotAdapt is an externally published ICLR 2025 paper that is also re-trained and re-evaluated here as a baseline under the same schedule, so it constitutes independent support rather than a circular justification. The temporal-coherence claim rests on an unmeasured assumption about slot identity stability across frames, which is a correctness or evaluation gap, not a circularity: FVD and the qualitative multi-frame results are computed independently of the slot-identity mechanism. No uniqueness theorem, ansatz, or benchmark protocol is imported from same-author work to forbid alternatives or force a result by construction. The paper is therefore self-contained against external benchmarks, and the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new entities are introduced; register tokens predate this paper (SlotAdapt; Darcet et al.) and the temporal aggregators are architectural modules. The scientific load is carried by domain assumptions about DINOv2 features, the frozen diffusion prior, the per-frame decoding strategy, and the training-set evaluation protocol.

free parameters (3)
  • Number of slots K = 7
    Hand-chosen following prior work (Appendix A.2); segmentation and generation depend on this capacity.
  • Temporal context window L = 5 frames (2 past, 2 future)
    Taken from SOLV (Sec 5.2); controls how much temporal context the aggregators see.
  • ISA scale multiplier delta = 5
    Inherited from SOLV (Appendix A.3); affects the relative-position encoding strength in Invariant Slot Attention.
assumptions (4)
  • domain assumption DINOv2 ViT-B/14 features encode enough object-level semantic information for unsupervised slot discovery on real-world videos
    Invoked throughout Sec 4.1; the encoder is a frozen backbone, so all object semantics are assumed present in these features.
  • domain assumption A frozen Stable Diffusion v1.5, augmented only with adapters, can faithfully reconstruct video frames conditioned on slots and register tokens
    Sec 4.2; the paper never retrains the diffusion model, so the transferability of its prior to slot conditioning is assumed.
  • domain assumption Per-frame decoding with Hungarian-matched slots yields temporally coherent video
    Sec 4.2 Inference; no joint temporal denoising is used, so coherence is assumed to come from slot alignment.
  • ad hoc to paper The 300-video subset of the YTVIS19 training set is a valid evaluation set for video generation metrics
    Sec 5.1 and Appendix A.1; the model was trained on the full training set including these videos, which inflates reconstruction-style metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Compositional Video Synthesis by Temporal Object-Centric Learning." pith.science (2026). https://pith.science/paper/NKXMZ4OQ

@misc{pith2026250720855,
  author       = {Pith},
  title        = {Pith review of: Compositional Video Synthesis by Temporal Object-Centric Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NKXMZ4OQ}},
  note         = {Machine review of arXiv:2507.20855}
}
read the original abstract

We present a novel framework for compositional video synthesis that leverages temporally consistent object-centric representations, extending our previous work, SlotAdapt, from images to video. While existing object-centric approaches either lack generative capabilities entirely or treat video sequences holistically, thus neglecting explicit object-level structure, our approach explicitly captures temporal dynamics by learning pose invariant object-centric slots and conditioning them on pretrained diffusion models. This design enables high-quality, pixel-level video synthesis with superior temporal coherence, and offers intuitive compositional editing capabilities such as object insertion, deletion, or replacement, maintaining consistent object identities across frames. Extensive experiments demonstrate that our method sets new benchmarks in video generation quality and temporal consistency, outperforming previous object-centric generative methods. Although our segmentation performance closely matches state-of-the-art methods, our approach uniquely integrates this capability with robust generative performance, significantly advancing interactive and controllable video generation and opening new possibilities for advanced content creation, semantic editing, and dynamic scene understanding.

Figures

Figures reproduced from arXiv: 2507.20855 by the authors.

Figure 1
Figure 1. Architecture Block Diagram We extract object-centric and temporally consistent information from input video frames using a visual backbone composed of DINOv2 and Invariant Slot Attention (ISA). The ISA mechanism generates slots for each frame, which are then aggregated temporally using a Transformer-based temporal aggregator to produce enriched, temporally-aware video slots. Concurrently, global context information … view at source ↗
Figure 2
Figure 2. Pose Invariance in Diffusion Conditioning. Comparison of video frame generation with and without register tokens on YTVIS dataset. Without register tokens (middle), objects appear in incorrect positions and backgrounds deviate from ground truth. With register to￾kens (right), generations accurately match ground truth (left), confirming that register tokens handle pose information while slots maintain object identity… view at source ↗
Figure 4
Figure 4. Segmentation Results. Qualitative comparison of video object segmentation on YTVIS (rows 1-2) and DAVIS17 (rows 3-4). Our method successfully delineates objects with accurate boundaries across diverse challenging scenarios. Row 1 shows segmentation of a bird with detailed boundary preservation, row 2 demonstrates segmentation of a monkey that covers most of the frame, row 3 shows segmentation of an animal against a … view at source ↗
Figures from the paper (21 more)
Figure 5
Figure 5. Figure 5: Compositional Generation Results. Demonstration of object-level editing capabilities through deletion and replacement operations on YTVIS (columns 1-3) and DAVIS17 (columns 4-5). Top rows show ground truth frames; bottom rows display edited results. Our method handles …
Figure 6
Figure 6. Figure 6: Temporal video generation results with and without register tokens on YTVIS dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Temporal video generation results with and without register tokens on YTVIS dataset. Results without register tokens (bottom) lead to spatial [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Temporal video generation results with and without register tokens on YTVIS dataset. Results without register tokens (bottom) lead to spatial [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Temporal video generation results with and without register tokens on YTVIS dataset. Results without register tokens (bottom) lead to spatial [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Temporal video generation results with and without register tokens on YTVIS dataset. Results without register tokens (bottom) lead to [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Temporal video generation results with and without register tokens on YTVIS dataset. Results without register tokens (bottom) lead to [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Temporal video generation results with and without register tokens on YTVIS dataset. Results without register tokens (bottom) lead to [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Temporal video generation results with and without register tokens on YTVIS dataset. Results without register tokens (bottom) lead to [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Temporal video generation results with and without register tokens on YTVIS dataset. Results without register tokens (bottom) lead to [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Temporal Video Generation on YTVIS Dataset. Multi-frame generation results showing object identity preservation and spatial coherence across five consecutive frames [PITH_FULL_IMAGE:figures/full_fig_p021_15.png]
Figure 16
Figure 16. Figure 16: Temporal Video Generation on YTVIS Dataset. Video generation on challenging sequences with multiple objects and complex backgrounds [PITH_FULL_IMAGE:figures/full_fig_p022_16.png]
Figure 17
Figure 17. Figure 17: Temporal Video Generation on DAVIS17 Dataset. Results demonstrating object motion tracking and spatial layout consistency over time [PITH_FULL_IMAGE:figures/full_fig_p023_17.png]
Figure 18
Figure 18. Figure 18: Temporal Video Generation on DAVIS17 Dataset. Generation results highlighting fine-detail preservation throughout temporal sequences [PITH_FULL_IMAGE:figures/full_fig_p024_18.png]
Figure 19
Figure 19. Figure 19: Segmentation Results on YTVIS. Consistent object boundary detection and identity preservation across dynamic motion and pose changes [PITH_FULL_IMAGE:figures/full_fig_p026_19.png]
Figure 20
Figure 20. Figure 20: Segmentation Results on YTVIS. Handling of significant shape and appearance variations with temporal tracking [PITH_FULL_IMAGE:figures/full_fig_p027_20.png]
Figure 21
Figure 21. Figure 21: Segmentation Results on DAVIS17. Tracking and segmentation of small, fast-moving objects through rapid motion and scale changes [PITH_FULL_IMAGE:figures/full_fig_p028_21.png]
Figure 22
Figure 22. Figure 22: Segmentation Results on DAVIS17. Distinguishing and tracking multiple closely positioned objects with stable segmentation masks [PITH_FULL_IMAGE:figures/full_fig_p029_22.png]
Figure 23
Figure 23. Figure 23: Compositional Editing Examples. Targeted object removal while maintaining scene coherence and temporal consistency [PITH_FULL_IMAGE:figures/full_fig_p031_23.png]
Figure 24
Figure 24. Figure 24: Compositional Editing Examples. Object deletion across diverse video sequences with realistic scene completion [PITH_FULL_IMAGE:figures/full_fig_p032_24.png]
Figure 25
Figure 25. Figure 25: Compositional Editing Examples. Targeted object replacement is performed while preserving scene coherence and temporal consistency. Top: the white dog is replaced with a black dog. Middle and bottom: the motorcycle is replaced with a black car [PITH_FULL_IMAGE:figure…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

73 extracted references · 67 canonical work pages

  1. [1]

    Slamp: Stochastic latent appearance and motion pre- diction

    Adil Kaan Akan, Erkut Erdem, Aykut Erdem, and Fatma G ¨uney. Slamp: Stochastic latent appearance and motion pre- diction. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2021

    work page 2021

  2. [2]

    Stretchbev: Stretching future instance prediction spatially and temporally

    Adil Kaan Akan and Fatma G ¨uney. Stretchbev: Stretching future instance prediction spatially and temporally. In Proc. of the European Conf. on Computer Vision (ECCV), 2022

    work page 2022

  3. [3]

    Slot-guided adaptation of pre-trained diffusion models for object-centric learning and compositional generation

    Adil Kaan Akan and Yucel Yemez. Slot-guided adaptation of pre-trained diffusion models for object-centric learning and compositional generation. In Proc. of the International Conf. on Learning Representations (ICLR), 2025

    work page 2025

  4. [4]

    Self- supervised Object-centric Learning for Videos

    G ¨orkay Aydemir, Weidi Xie, and Fatma G ¨uney. Self- supervised Object-centric Learning for Videos. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    work page 2023

  5. [5]

    Systematic generalization: What is required and can it be learned? In Proc

    Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, and Aaron Courville. Systematic generalization: What is required and can it be learned? In Proc. of the International Conf. on Learning Repre- sentations (ICLR), 2019

    work page 2019

  6. [6]

    Object discovery from motion- guided tokens

    Zhipeng Bao, Pavel Tokmakov, Yu-Xiong Wang, Adrien Gaidon, and Martial Hebert. Object discovery from motion- guided tokens. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2023

    work page 2023

  7. [7]

    Lumiere: A space-time diffusion model for video generation

    Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, et al. Lumiere: A space-time diffusion model for video generation. In SIGGRAPH Asia 2024 Conference Papers, 2024

    2024

  8. [8]

    Invariant slot attention: Object discovery with slot-centric reference frames

    Ondrej Biza, Sjoerd Van Steenkiste, Mehdi SM Sajjadi, Gamaleldin Fathy Elsayed, Aravindh Mahendran, and Thomas Kipf. Invariant slot attention: Object discovery with slot-centric reference frames. InProc. of the International Conf. on Machine Learning (ICML), 2023

    work page 2023

Show all 73 references
  1. [9]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  2. [10]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2021

    2021

  3. [11]

    Pixart-α: Fast training of diffusion trans- former for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-α: Fast training of diffusion trans- former for photorealistic text-to-image synthesis. In Proc. of the International Conf. on Learning Represen...

    2024

  4. [12]

    Vision transformers need registers

    Timoth ´ee Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In Proc. of the International Conf. on Learning Representations (ICLR) , 2024

    2024

  5. [13]

    Diffusion models beat GANs on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. In Advances in Neural Infor- mation Processing Systems (NeurIPS), 2021

    2021

  6. [14]

    Betrayed by attention: A simple yet ef- fective approach for self-supervised video object segmenta- tion

    Shuangrui Ding, Rui Qian, Haohang Xu, Dahua Lin, and Hongkai Xiong. Betrayed by attention: A simple yet ef- fective approach for self-supervised video object segmenta- tion. In Proc. of the European Conf. on Computer Vision (ECCV), 2024

    2024

  7. [15]

    SAVi++: Towards end-to-end object-centric learning from real-world videos

    Gamaleldin Fathy Elsayed, Aravindh Mahendran, Sjoerd van Steenkiste, Klaus Greff, Michael Curtis Mozer, and Thomas Kipf. SAVi++: Towards end-to-end object-centric learning from real-world videos. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  8. [16]

    Attend, infer, re- peat: Fast scene understanding with generative models

    SM Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, David Szepesvari, Geoffrey E Hinton, et al. Attend, infer, re- peat: Fast scene understanding with generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2016

    2016

  9. [17]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Proc. of the International Conf. on Machine Le...

    2024

  10. [18]

    The PASCAL visual object classes (VOC) challenge

    Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The PASCAL visual object classes (VOC) challenge. International Journal of Com- puter Vision (IJCV), 2010

    2010

  11. [19]

    Connectionism and cognitive architecture: A critical analysis

    Jerry A Fodor and Zenon W Pylyshyn. Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1- 2):3–71, 1988

    1988

  12. [20]

    Understanding the diffi- culty of training deep feedforward neural networks

    Xavier Glorot and Yoshua Bengio. Understanding the diffi- culty of training deep feedforward neural networks. InProc. of the International Conf. on Artificial Intelligence and Statistics (AISTATS), 2010

    2010

  13. [21]

    Multi-object representation learning with iterative variational inference

    Klaus Greff, Rapha ¨el Lopez Kaufman, Rishabh Kabra, Nick Watters, Christopher Burgess, Daniel Zoran, Loic Matthey, Matthew Botvinick, and Alexander Lerchner. Multi-object representation learning with iterative variational inference. JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. ...

    2020

  14. [22]

    On the binding problem in artificial neural networks

    Klaus Greff, Sjoerd Van Steenkiste, and J ¨urgen Schmidhu- ber. On the binding problem in artificial neural networks. arXiv preprint arXiv:2012.05208, 2020

    2012 arXiv

  15. [23]

    Gans trained by a two time-scale update rule converge to a local nash equi- librium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equi- librium. In Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  16. [24]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  17. [25]

    Imagen video: High definition video generation with diffusion models

    Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi, and Tim Salimans. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022

    2022 arXiv

  18. [26]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  19. [27]

    Object-centric slot diffusion

    Jindong Jiang, Fei Deng, Gautam Singh, and Sungjin Ahn. Object-centric slot diffusion. In Advances in Neural Informa- tion Processing Systems (NeurIPS), 2023

    2023

  20. [28]

    CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) , 2017

    2017

  21. [29]

    ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation

    Laurynas Karazija, Iro Laina, and Christian Rupprecht. ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation. In In Advances of Neural Infor- mation Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021

    2021

  22. [30]

    Con- ditional object-centric learning from video

    Thomas Kipf, Gamaleldin Fathy Elsayed, Aravindh Ma- hendran, Austin Stone, Sara Sabour, Georg Heigold, Rico Jonschkowski, Alexey Dosovitskiy, and Klaus Greff. Con- ditional object-centric learning from video. In Proc. of the International Conf. on Learning Representations (ICLR), 2022

    2022

  23. [31]

    Sequential attend, infer, repeat: Generative mod- elling of moving objects

    Adam Kosiorek, Hyunjik Kim, Yee Whye Teh, and Ingmar Posner. Sequential attend, infer, repeat: Generative mod- elling of moving objects. In Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  24. [32]

    Structured object-aware physics prediction for video modeling and planning

    Jannik Kossen, Karl Stelzner, Marcel Hussing, Claas Voel- cker, and Kristian Kersting. Structured object-aware physics prediction for video modeling and planning. In Proc. of the International Conf. on Learning Representations (ICLR), 2019

    2019

  25. [33]

    Hierarchical compact clustering attention (coca) for unsupervised object-centric learning

    Can Kucuksozen and Yucel Yemez. Hierarchical compact clustering attention (coca) for unsupervised object-centric learning. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  26. [34]

    Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux, 2024

    2024

  27. [35]

    Building machines that learn and think like people

    Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. Building machines that learn and think like people. Behavioral and Brain Sciences, 40, 2017

    2017

  28. [36]

    Fuxin Li, Taeyoung Kim, Ahmad Humayun, David Tsai, and James M. Rehg. Video segmentation by tracking many figure-ground segments. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013

    2013

  29. [37]

    Microsoft COCO: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft COCO: Common objects in context. In Proc. of the European Conf. on Computer Vision (ECCV), 2014

    2014

  30. [38]

    Improving generative imagination in object-centric world models

    Zhixuan Lin, Yi-Fu Wu, Skand Peri, Bofeng Fu, Jindong Jiang, and Sungjin Ahn. Improving generative imagination in object-centric world models. In Proc. of the International Conf. on Machine Learning (ICML), 2020

    2020

  31. [39]

    Object- centric learning with slot attention

    Francesco Locatello, Dirk Weissenborn, Thomas Un- terthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object- centric learning with slot attention. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  32. [40]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Proc. of the International Conf. on Learning Representations (ICLR), 2019

    2019

  33. [41]

    Temporally consistent object-centric learning by contrasting slots

    Anna Manasyan, Maximilian Seitzer, Filip Radovic, Georg Martius, and Andrii Zadaianchuk. Temporally consistent object-centric learning by contrasting slots. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) , 2025

    2025

  34. [42]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to- image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to- image diffusion models. In Proc. of the Conf. on Artificial Intelligence (AAAI), 2024

    2024

  35. [43]

    Segmentation of moving objects by long term video analysis

    Peter Ochs, Jitendra Malik, and Thomas Brox. Segmentation of moving objects by long term video analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence (P AMI), 2013

    2013

  36. [44]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. In Transactions on Machine Learning Research ...

    2023

  37. [45]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  38. [46]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In Proc. of the International Conf. on Learning Representations (ICLR), 2024

    2024

  39. [47]

    Rethinking image-to-video adaptation: An object-centric perspective

    Rui Qian, Shuangrui Ding, and Dahua Lin. Rethinking image-to-video adaptation: An object-centric perspective. In Proc. of the European Conf. on Computer Vision (ECCV) , 2024

    2024

  40. [48]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proc. of the Inte...

    2021

  41. [49]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  42. [50]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  43. [51]

    Photorealistic text-to-image JOURNAL OF LATEX CLASS FILES, VOL

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 12 diffusi...

    2020

  44. [52]

    Toward causal representation learning

    Bernhard Sch ¨olkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. IEEE Trans. on Pattern Analysis and Machine Intelligence (P AMI), 109(5):612–634, 2021

    2021

  45. [53]

    Bridging the gap to real-world object-centric learning

    Maximilian Seitzer, Max Horn, Andrii Zadaianchuk, Do- minik Zietlow, Tianjun Xiao, Carl-Johann Simon-Gabriel, Tong He, Zheng Zhang, Bernhard Sch¨olkopf, Thomas Brox, et al. Bridging the gap to real-world object-centric learning. In Proc. of the International Conf. on Learning ...

    2023

  46. [54]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. In Proc. of the International Conf. on Learning Representations (ICLR), 2023

    2023

  47. [55]

    Illiterate dall- e learns to compose

    Gautam Singh, Fei Deng, and Sungjin Ahn. Illiterate dall- e learns to compose. In Proc. of the International Conf. on Learning Representations (ICLR), 2021

    2021

  48. [56]

    Simple unsu- pervised object-centric learning for complex and natural- istic videos

    Gautam Singh, Yi-Fu Wu, and Sungjin Ahn. Simple unsu- pervised object-centric learning for complex and natural- istic videos. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  49. [57]

    Guided latent slot diffusion for object-centric learning

    Krishnakant Singh, Simone Schaub-Meyer, and Stefan Roth. Guided latent slot diffusion for object-centric learning. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  50. [58]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proc. of the International Conf. on Machine Learning (ICML), 2015

    2015

  51. [59]

    Core knowl- edge

    Elizabeth S Spelke and Katherine D Kinzler. Core knowl- edge. Developmental Science, 2007

    2007

  52. [60]

    Mind games: Game engines as an architecture for intuitive physics

    Tomer D Ullman, Elizabeth Spelke, Peter Battaglia, and Joshua B Tenenbaum. Mind games: Game engines as an architecture for intuitive physics. Trends in cognitive sciences, 21(9):649–665, 2017

    2017

  53. [61]

    To- wards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018

    2018 arXiv

  54. [62]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  55. [63]

    Phenaki: Variable length video generation from open domain textual descrip- tions

    Ruben Villegas, Zalan Borsos, Aditya Ramesh, Jiahui Li, Jacob Menick, Alexander Kirillov, Oriol Vinyals, Aaron van den Oord, Nal Kalchbrenner, et al. Phenaki: Variable length video generation from open domain textual descrip- tions. In Proc. of the International Conf. on Learn...

    2023

  56. [64]

    Videocomposer: Compositional video syn- thesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou. Videocomposer: Compositional video syn- thesis with motion controllability. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  57. [65]

    Bovik, Hamid R

    Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P . Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing , 13(4):600–612, 2004

    2004

  58. [66]

    Tune-a-video: One-shot tun- ing of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tun- ing of image diffusion models for text-to-video generation. In Proc. of the IEEE International Conf. on Computer ...

    2023

  59. [67]

    SlotFormer: Unsupervised visual dynamics simulation with object-centric models

    Ziyi Wu, Nikita Dvornik, Klaus Greff, Thomas Kipf, and Animesh Garg. SlotFormer: Unsupervised visual dynamics simulation with object-centric models. In Proc. of the Inter- national Conf. on Learning Representations (ICLR), 2023

    2023

  60. [68]

    Slotdiffusion: Object-centric generative model- ing with diffusion models

    Ziyi Wu, Jingyu Hu, Wuyue Lu, Igor Gilitschenski, and Ani- mesh Garg. Slotdiffusion: Object-centric generative model- ing with diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  61. [69]

    Segment- ing moving objects via an object-centric layered represen- tation

    Junyu Xie, Weidi Xie, and Andrew Zisserman. Segment- ing moving objects via an object-centric layered represen- tation. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  62. [70]

    Youtube-vos: A large-scale video object segmentation benchmark

    Ning Xu, Linjie Yang, Yuchen Fan, Dingcheng Yue, Yuchen Liang, Jianchao Yang, and Thomas Huang. Youtube-vos: A large-scale video object segmentation benchmark. In Proc. of the European Conf. on Computer Vision (ECCV), 2018

    2018

  63. [71]

    Video instance segmentation

    Linjie Yang, Yuchen Fan, and Ning Xu. Video instance segmentation. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019

    2019

  64. [72]

    Efros, Eli Shechtman, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018

    2018

  65. [73]

    Controlvideo: Training-free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. In Proc. of the International Conf. on Learning Representations (ICLR) , 2024. Adil Kaan Akan received the B.Sc. degree in co...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.