{"id":"eb127ff5-4432-42ff-8469-b64d4d54a12e","arxiv_id":"2412.14295","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Slot-slot temporal contrast improves temporal consistency and object discovery in unsupervised object-centric video models, reaching state-of-the-art FG-ARI on MOVi-E and YouTube-VIS.","lead":"The authors add a contrastive loss that keeps the same slot attached to the same object across video frames, while pushing different slots apart. This makes unsupervised object-centric video models more temporally consistent and improves object discovery on synthetic and real-world videos.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 5's positive pairs assume slot index equals object identity; the paper never directly measures this, so the temporal-consistency mechanism could be training against wrong pairs after occlusions or reappearances.","rationale":"The reader's weakest assumption is exactly the slot-index-to-object-identity correspondence, and I agree that this is the most load-bearing point. The paper has independent support that partially mitigates the concern: per-frame image FG-ARI improves (so the loss does help discovery even without temporal identity), the dynamics-prediction experiment would likely fail under severe identity instability, and the qualitative results show stable color-coded tracking in several videos. However, none of these directly measures the identity-preservation assumption that Eqs. 4–5 rely on. The contrastive loss is the novel mechanism; if it is pairing wrong slots after occlusions, then the reported video-FG-ARI gains could arise from a different, potentially unintended behavior (e.g., slot collapse mitigated by reconstruction, or the learned initialization doing most of the work). The proposed test is straightforward to run on the existing checkpoints and would settle the issue. I also note the abstract's 'state-of-the-art on both synthetic and real-world datasets' is an overclaim given that Table 1 shows lower mBO than VideoSAUR on both MOVi-C and MOVi-E; but this is a presentation issue, not the core mechanism, so it does not change my verdict. Overall, the reader's conditional verdict is appropriate: the method is promising and well evaluated, but this one untested assumption justifies the condition.","tokens_in":19546,"tokens_out":4399,"duration_ms":42704,"concrete_test":"Compute a direct slot-object identity-stability metric. For every frame t and slot k on the MOVi-C/MOVi-E validation sets and YTVIS, match each slot to the GT object with maximum mask IoU (one-to-one via Hungarian), then define stability(t,k) = 1 if slot k's matched object at t equals its matched object at t−1, else 0. Report mean stability over all slots, frames, and videos for SLOT CONTRAST and for VideoSAUR/VideoSAURv2, separately for the 60-sequence occlusion subset. If mean stability is below, say, 0.95 on the occlusion subset or shows a material gap versus VideoSAUR, the index-pairing assumption fails precisely where it matters; then the video FG-ARI numbers cannot be attributed to the intended contrastive mechanism, and a retraining with oracle object-matched positives (or a permutation-invariant loss) would be needed to establish the claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that SLOT CONTRAST produces temporally consistent object-centric representations—rests on the positive-pair definition in Eqs. 4–5: slot i at t−1 is paired with slot i at t. This is valid only if the recurrent slot attention module keeps slot indices aligned with object identity over time. Nothing in the training objective measures object identity; the loss is purely index-based. If the grouper permutes slots after an occlusion or reappearance, the contrastive loss actively repels the representation of the true same object (now at a different index) and attracts two different objects, and gradients from this erroneous pairing propagate into the encoder, predictor, and initialization. Appendix D explicitly acknowledges this risk ('wrong slots pulling') and argues that the feature reconstruction loss prevents it, but that is a heuristic claim, not a measurement. The evaluation metrics are aggregate: video FG-ARI does penalize identity switches, but it is averaged over all slots and frames and over easy and hard videos, so occasional swaps—especially on the exact occlusion/reappearance cases the method claims to handle—could be masked. The 60-sequence occlusion subset reports only mBO on occluded objects, not whether the slot index preserved the same object's identity. Thus the load-bearing assumption is unverified, and the method's stated mechanism (contrastive attraction of same-object slots) is not directly confirmed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SLOT CONTRAST, an unsupervised video object-centric learning method that adds a slot-slot temporal contrastive loss to a recurrent slot-attention architecture built on DINOv2 features. The loss pulls same-index slots from adjacent frames together and repels all other slots in the batch (Eqs. 4-5), with a learned slot initialization replacing random initialization. The authors report experiments on MOVi-C, MOVi-E, and YouTube-VIS 2021, showing improved video FG-ARI over VideoSAUR, improved per-frame FG-ARI on MOVi-E (84.8) and YTVIS, improved downstream object dynamics prediction with SlotFormer on MOVi-C, and better mBO on an occlusion subset of MOVi-C. The paper claims state-of-the-art object discovery and temporal consistency, including superiority over weakly supervised methods that use motion masks.","tokens_in":19940,"tokens_out":7685,"duration_ms":66432,"significance":"If the results hold, the method is a simple and effective contribution to real-world video object-centric learning: the per-frame FG-ARI of 84.8 on MOVi-E and video FG-ARI of 38.0 on YTVIS are strong, and the downstream dynamics prediction gain on MOVi-C (63.8 vs 50.7 FG-ARI) is notable. The paper is thorough in providing ablations, hyperparameter tables, and qualitative visualizations, and the proposed loss is easy to integrate into existing slot-based video models. However, the central temporal-consistency mechanism rests on an assumption about slot-index stability that is never directly measured, and the reported numbers lack multiple seeds or error bars. The claims in the abstract are also stronger than the evidence for synthetic datasets, where mBO is lower than VideoSAUR.","major_comments":[{"comment":"The positive-pair construction assumes that slot i at time t-1 and slot i at time t represent the same physical object. This assumption is load-bearing for the claimed mechanism, but it is not directly verified. The reported metrics (video FG-ARI, mBO) are aggregate over all slots and frames, and the occlusion study in App. K uses only 60 filtered sequences and reports only mBO, not whether the slot index preserved the same object's identity through occlusion and reappearance. Appendix D acknowledges the 'wrong slots pulling' risk and argues that the feature reconstruction loss prevents it, but that is a heuristic argument, not an evaluation. I ask the authors to directly measure slot-object identity preservation, for example by reporting the frequency of slot-index switches for ground-truth objects matched across frames, especially on the occlusion subset and on YTVIS, or by testing the loss under a deliberately permuted slot order between frames. Without such a measurement, the improved temporal consistency could be partly an artifact of optimizing the same index-stability quantity measured by video FG-ARI.","section":"Section 3.2, Eqs. (4)-(5); Appendix D; Appendix K"},{"comment":"No standard deviations or number of seeds are reported for any experiment. Several key comparisons are small or dataset-dependent: in Table 4, the YTVIS mBO values for the three loss configurations are 31.4, 33.6, and 33.7, and the MOVi-E mBO values are 28.4, 29.1, and 29.2; in Table 3, MOVi-E FG-ARI is 70.6 vs 70.5 for the dynamics prediction comparison. Without repeated seeds and variance estimates, the reader cannot judge whether the claimed improvements are robust, especially where the method is worse on mBO than VideoSAUR (Table 1). I recommend reporting at least 3-5 seeds with mean and standard deviation for all main tables.","section":"Tables 1-5"},{"comment":"The abstract's claim of 'state-of-the-art results on both synthetic and real-world datasets' is stronger than the evidence. In Table 1, VideoSAUR achieves higher mBO than SLOT CONTRAST on both MOVi-C (38.9 vs 32.7) and MOVi-E (35.6 vs 29.2), so the method is not state-of-the-art on mask quality for synthetic data. The claim of 'outperforming even weakly-supervised methods that leverage motion masks' is based only on per-frame image FG-ARI in Table 2, not on video metrics or mBO, and the margin over DIOD is 2.6 points. Please qualify the claims to the specific metrics and datasets where they hold, and note the mBO trade-off explicitly.","section":"Abstract; Section 4.1; Table 1; Table 2"}],"minor_comments":[{"comment":"The notation for the decoder output is inconsistent: Section 3.1 says the decoder produces reconstructions \\hat g_t, while Eq. (6) writes Lrec(h_t, \\hat h_t). Please unify the notation.","section":"Section 3.1 and Eq. (6)"},{"comment":"The implementation described in Appendix C, which applies cross-entropy between P = softmax(A) and the identity matrix, includes the positive pair in the denominator of the softmax. This differs from Eqs. (4)-(5), where the indicator explicitly excludes the positive pair from the denominator. Please align the description with the equations or explain the equivalence.","section":"Appendix C and Eqs. (4)-(5)"},{"comment":"The sentence 'Using the intra-video contrastive loss yields improvements over the feature reconstruction baseline (+5.1 FG-ARI and +1.5 mBO on MOVi-C)' is dataset-specific but reads as a general statement. In Table 4, the intra-video loss decreases FG-ARI on MOVi-E from 79.8 to 78.7. Please report the effect per dataset.","section":"Section 4.3, Table 4"},{"comment":"The text says 'Our method outperforms all state-of-the-art approaches by a significant margin, with the sole exception of VideoSAUR, where we observe a minor performance gap of just 0.4 points.' The phrase 'significant margin' is contradicted by the 0.4-point gap to VideoSAUR on the same metric; please rephrase.","section":"Appendix H"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid incremental contribution to video object-centric learning, and the real-world YTVIS improvements are interesting. The main weakness is that the central temporal-consistency mechanism is verified only indirectly; adding a direct identity-switch metric and error bars would substantially strengthen the paper. The abstract and text should be toned down to match the mBO results on synthetic datasets. I do not see a fundamental flaw that would require rejection, but the missing evidence is load-bearing for the core claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one if you work on unsupervised object-centric video. The core is simple: apply an InfoNCE loss to slots across adjacent frames, using all other slots in the batch as negatives, plus a learned first-frame initialization. It is a natural idea, but I do not know prior work that did exactly this, and the paper works the idea hard: three datasets, a DINOv2-trained VideoSAUR baseline, ablations, dynamics prediction, an occlusion subset, a SAM2 comparison, and explicit failure cases. Credit where due: the per-frame FG-ARI gain on MOVi-E (84.8 vs 80.8 for SOLV, 78.4 for VideoSAUR) is real and not directly optimized by the loss, and the YouTube-VIS video FG-ARI jump (+6.8 over VideoSAURv2) is the strongest evidence. The dynamics prediction numbers are modest but move in the right direction.\n\nSoft spots: no error bars or seeds, and some differences, like the MOVi-C image FG-ARI of 75.1 vs VideoSAUR's 75.5, are within noise. The abstract's \"state-of-the-art on synthetic\" overstates Table 1, where mBO is below VideoSAUR on both MOVi-C and MOVi-E; the claim only holds for FG-ARI and per-frame FG-ARI. The bigger conceptual issue is the one the stress-test flagged: Eq. 5 defines positives by slot index, not object identity. If the grouper permutes slots after an occlusion, the loss actively trains against the correct correspondence. The paper acknowledges this in App. D and argues reconstruction prevents \"wrong slots pulling,\" which is plausible but not measured. The occlusion experiment (60 sequences) reports mBO on occluded objects, not whether the slot index stayed with the same object; video FG-ARI would penalize identity switches, but it is averaged over easy and hard videos. This is a real gap, but not fatal: the consistent gains across per-frame FG-ARI, video FG-ARI, and dynamics suggest the objective usually does what it claims, and the loss is simple enough that a referee can probe it directly.\n\nNo code is released, which makes the missing seeds worse for reproducibility. Still, this deserves a serious referee. I would ask the referees to demand error bars and a direct identity-preservation check: slot correspondence versus ground-truth object ID through occlusions and reappearances. If that holds, it is a solid subfield contribution.","headline":"A simple, well-executed batch slot-slot InfoNCE objective that improves temporal consistency and per-frame object discovery; the main unverified assumption is that slot index tracks object identity across occlusions.","tokens_in":20337,"tokens_out":2228,"would_cite":true,"duration_ms":21501,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A temporal contrastive loss on slots makes unsupervised video object discovery state of the art.","keywords":["object-centric learning","temporal consistency","slot attention","contrastive learning","slot-slot contrastive loss","unsupervised object discovery","video representation learning","DINOv2 features"],"falsifier":"Take video sequences with ground-truth object identities and compute, for every pair of consecutive frames, the best assignment of slots to objects; then measure the slot-identity switch rate, the fraction of slot indices whose assigned object changes without an occlusion event. If the model trained with SLOT CONTRAST has a switch rate comparable to the feature-reconstruction baseline, the temporal-consistency claim is unsupported. A second check is to randomly permute slot indices between frames during training: if the reported FG-ARI gains on YouTube-VIS do not collapse, the improvements come from something other than temporal binding.","tokens_in":19346,"feed_emoji":"🎯","tokens_out":7236,"duration_ms":59930,"temperature":0.7,"pith_summary":"Unsupervised object-centric learning aims to decompose videos into stable object representations without labels, but existing slot-based models often let slots drift or swap objects over time. This paper argues that the missing piece is an explicit training objective for temporal consistency: a contrastive loss that pulls a slot's representation in one frame toward the same-index slot in the next frame while pushing it away from every other slot in the batch. The claim is that this inductive bias does two things at once—it keeps slots tied to the same physical object across time, and it sharpens object discovery in individual frames. On MOVi-C, MOVi-E, and YouTube-VIS the method reaches state-of-the-art video FG-ARI and outperforms weakly supervised methods that use motion masks. If right, this makes unsupervised representations usable for tracking through occlusions and for learning object dynamics.","feed_headline":"Slot-slot contrast loss beats motion-supervised video object discovery","feed_subtitle":"Unsupervised model keeps slots locked to objects across frames, improving tracking and dynamics prediction on real video.","key_machinery":"The load-bearing mechanism is the slot-slot contrastive loss (Eq. 5), an InfoNCE loss over cosine similarities between slot vectors from adjacent frames: the positive sample for slot $i$ in video $j$ at time $t-1$ is the same-index slot at time $t$, and the negatives are all other slots in the batch. It is paired with a learned fixed initialization $S_0$ that makes first-frame slots mutually contrastive, and with a feature reconstruction loss on DINOv2 features that keeps each slot informative about its input region. The loss works by turning slot identity into a learnable object identifier, while the batch-wide negative set prevents the model from satisfying consistency merely through a distinctive initialization.","core_discovery":"The paper's central claim is that temporal consistency in video object-centric models can be enforced directly at the slot level, and that doing so improves both consistency and discovery. Slot representations $s_{t-1}^i$ and $s_t^i$ from consecutive frames form positive pairs; the denominator of an InfoNCE-style loss includes all other slots in the batch, from the same video and from other videos. The authors report that this yields 84.8 image FG-ARI on MOVi-E and 38.0 video FG-ARI on YouTube-VIS, surpassing VideoSAUR and even weakly supervised models that consume motion masks. They further show the learned slots support downstream dynamics prediction with SlotFormer and improve re-identification after full occlusions, presenting the contrastive objective as a simple replacement for hand-crafted temporal regularizers in object-centric video learning.","pith_inferences":["The paper never verifies that slot index $i$ actually tracks the same physical object; it reports aggregate mask metrics and a 60-sequence occlusion subset. A direct identity-switch metric would make the temporal-consistency claim falsifiable and could reveal whether the loss works through genuine object permanence or through stabilizing mask geometry.","Because the negative set spans the whole batch, contrastive difficulty grows with batch size; a testable extension is whether larger batches or hard-negative mining over similar-looking objects further improve consistency.","The loss constrains the slot space only, not the decoder, so it should transfer to other slot-based video models, including diffusion-based object-centric approaches, provided they use recurrent or propagated slot representations.","The batch-wide contrast acts as an implicit slot-competition regularizer, which may explain why redundant slots shut down and the predicted number of objects matches the ground-truth distribution more closely without an explicit sparsity prior."],"forward_implications":["If the central claim holds, unsupervised object-centric video models can be made temporally consistent with a single contrastive term, requiring no motion masks and no human annotations.","The 84.8 image FG-ARI on MOVi-E suggests temporal contrast also improves per-frame segmentation quality, not just tracking stability.","On YouTube-VIS, video FG-ARI rises from 31.2 (VideoSAURv2) to 38.0, narrowing the gap to zero-shot supervised tracking with SAM2 when combined with SLOT CONTRAST masks.","Consistent slots improve downstream dynamics prediction with SlotFormer on MOVi-C (63.8 vs 50.7 FG-ARI) and give modest gains on YouTube-VIS.","Learned initialization matters: under the contrastive objective, fixed learned queries outperform random Gaussian initialization on all three datasets."],"supporting_citations":[{"why":"Provides the recurrent slot-attention video framework whose temporal-update design SLOT CONTRAST builds on.","marker":"[28]"},{"why":"VideoSAUR is the state-of-the-art baseline using temporal feature-similarity prediction; SLOT CONTRAST compares against it on all datasets.","marker":"[64]"},{"why":"Supplies the feature reconstruction objective and MLP decoder that keep slots informative in addition to contrastive.","marker":"[44]"},{"why":"Defines Slot Attention, the grouping module from which the recurrent grouping and slot representations derive.","marker":"[32]"},{"why":"Provides the frozen DINOv2 dense features that serve as input and reconstruction target.","marker":"[38]"},{"why":"Defines the InfoNCE contrastive loss whose form the slot-slot contrastive loss adopts.","marker":"[37]"},{"why":"SlotFormer is the dynamics module trained on learned slots to test downstream object dynamics prediction.","marker":"[56]"},{"why":"SOLV is an unsupervised video object-centric baseline that SLOT CONTRAST outperforms on per-frame object discovery.","marker":"[2]"}],"fun_headline_variants":["Contrastive slots keep objects locked across video frames","Slot contrast loss beats motion masks for video object discovery","Temporal contrast for slots: better video decomposition, no motion cues","Unsupervised video objects: contrastive slots trump motion supervision","Locking slots to objects: contrastive loss fixes temporal consistency"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The loss assumes that slot number $i$ in one frame and slot number $i$ in the next frame always correspond to the same object; if the slot-attention module reorders or reassigns slots at object splits, merges, or reappearances, the objective actively pulls unrelated slots together and can corrupt consistency.","fun_headline_variants_meta":{"raw":{"variants":["Contrastive slots keep objects locked across video frames","Slot contrast loss beats motion masks for video object discovery","Temporal contrast for slots: better video decomposition, no motion cues","Unsupervised video objects: contrastive slots trump motion supervision","Locking slots to objects: contrastive loss fixes temporal consistency"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0004,"raw_usage":{"total_tokens":2044,"prompt_tokens":852,"completion_tokens":1192,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":1109}},"tokens_in":468,"tokens_out":1192,"duration_ms":8764,"temperature":1.0,"reasoning_tokens":1109,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:20:58.743755+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take video sequences with ground-truth object identities and compute, for every pair of consecutive frames, the best assignment of slots to objects; then measure the slot-identity switch rate, the fraction of slot indices whose assigned object changes without an occlusion event. If the model trained with SLOT CONTRAST has a switch rate comparable to the feature-reconstruction baseline, the temporal-consistency claim is unsupported. A second check is to randomly permute slot indices between frames during training: if the reported FG-ARI gains on YouTube-VIS do not collapse, the improvements come from something other than temporal binding.","supporting_citations":[{"cited_title":"Conditional Object-centric Learning from Video","cited_arxiv_id":null,"evidence_quote":"Provides the recurrent slot-attention video framework whose temporal-update design SLOT CONTRAST builds on."},{"cited_title":"Object-centric learning for real-world videos by predicting temporal feature similarities","cited_arxiv_id":null,"evidence_quote":"VideoSAUR is the state-of-the-art baseline using temporal feature-similarity prediction; SLOT CONTRAST compares against it on all datasets."},{"cited_title":"Bridging the gap to real-world object- centric learning","cited_arxiv_id":null,"evidence_quote":"Supplies the feature reconstruction objective and MLP decoder that keep slots informative in addition to contrastive."},{"cited_title":"Object-Centric Learn- ing with Slot Attention","cited_arxiv_id":null,"evidence_quote":"Defines Slot Attention, the grouping module from which the recurrent grouping and slot representations derive."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the frozen DINOv2 dense features that serve as input and reconstruction target."},{"cited_title":"Slotformer: Unsupervised visual dynamics simulation with object-centric models","cited_arxiv_id":null,"evidence_quote":"SlotFormer is the dynamics module trained on learned slots to test downstream object dynamics prediction."},{"cited_title":"Self- supervised object-centric learning for videos","cited_arxiv_id":null,"evidence_quote":"SOLV is an unsupervised video object-centric baseline that SLOT CONTRAST outperforms on per-frame object discovery."}],"review_version":1}