Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

Smoothing Slot Attention Iterations and Recurrences

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Two small changes to Slot Attention—preheating queries with image features and trimming later-frame refinement steps—are claimed to deliver state-of-the-art object-centric learning on images and videos.

desk verdict A solid empirical study with overstated headline claims: the preheating and differentiated iteration counts are worth a look, but the SOTA and 'consistent boost' claims are contradicted by the paper's own tables. read the letter →

arxiv 2508.05417 v5 pith:XZF5JM37 submitted 2025-08-07 cs.CV

Rongzhen Zhao , Wenyan Yang , Juho Kannala , Joni Pajarinen This is my paper
classification cs.CV
keywords object-centriclearningslotattentionquerypreheatingself-distillationvideoobjectsegmentationvisualquestionansweringiterativerefinementcontractivefixedpoint
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that two small changes to Slot Attention remove two bottlenecks that object-centric learning models inherit. For a first image or first video frame, the queries that seed slot refinement carry no sample-specific information, so the paper preheats them with input features using a tiny module trained by self-distillation; this makes each refinement iteration start closer to the final slots. For later video frames, whose queries already contain information from previous slots, the paper argues that using the full three refinement iterations is wasteful and that one iteration works better. Together these changes—SmoothSA—are claimed to produce new state-of-the-art object discovery on image and video benchmarks, with consistent gains on object recognition and visual question answering.

What carries the argument

The load-bearing object is the preheater $f_P$: a single Transformer decoder block whose cross-attention comes before self-attention, inserted between query initialization and the Slot Attention module, trained by MSE self-distillation against the current slots with stop-gradient on the preheated queries. The second mechanism is the differentiated recursion: $3$ SA iterations on the first frame and $1$ on non-first frames while sharing weights. The theoretical engine is the observation that Slot Attention is a contraction with a fixed point, so starting closer to the fixed point—or taking fewer steps when already close—improves the refinement trajectory; the paper formalizes this with Lipsch

What would settle it

Corrupt the aggregated slots that supervise the preheater with noise of growing variance—beyond the 1k small corruptions already tested—and track object-discovery mIoU after each of the three SA iterations. If the preheated queries amplify the corruption, with error growing across iterations instead of decaying toward the fixed point, the central smoothing claim fails.

Watch

Extended reading notes

Core claim

Slot Attention treats queries as cold-start noise and refines them by repeated cross-attention against image features, applying the same transform to every video frame. The paper's central claim is that both choices are suboptimal. Cold-start queries lack sample-specific cues, so a tiny preheater—a single Transformer decoder block with cross-attention before self-attention—should first condition the queries on input features; trained by self-distillation against the slots and with stop-gradient, it moves the starting point closer to the fixed point of the Slot Attention contraction. The same three-iteration transform should not be reused on non-first video frames, where queries are already i

Load-bearing premise

The preheater learns from the Slot Attention module's own outputs, so if those slots are ever poor, the preheated queries inherit the error and can drive the slots even worse; the paper's stress test covers only small random corruptions, not sustained bad-slot regimes.

Editorial extensions

If this is right

  • If SmoothSA is correct, image and first-frame video object discovery should show high segmentation quality even at the first refinement iteration, not only after all three iterations.
  • If SmoothSA is correct, video object-centric models should use different refinement budgets for first and non-first frames; the 3-and-1 schedule beats both 1-and-1 and 3-and-3 in the paper's ablations.
  • If SmoothSA is correct, the improved slots transfer to downstream tasks: object recognition improves on COCO and YTVIS, and VQA improves on GQA and CLEVRER, with the upstream model frozen.
  • If SmoothSA is correct, the gains come at comparable or lower compute, since the preheater is tiny and later video frames use fewer SA iterations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the preheater is a general warm-start recipe—any Slot Attention style iterative refinement starting from sample-agnostic queries (Gaussian, codebook, or positional priors) could be preheated the same way, and the contraction argument implies the gain should be largest when the starting queries are least informative.
  • Beyond the paper: the fixed 3-versus-1 schedule is a natural place for an adaptive controller, since query informativeness varies with scene complexity and motion; the paper itself lists the fixed schedule as a limitation.
  • Beyond the paper: if the reported gains hold, SmoothSA slots become a drop-in upstream representation for video prediction and planning, though the paper evaluates only recognition and VQA; that is a direct testable extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SmoothSA, which modifies Slot Attention (SA) for image and video object-centric learning. Two mechanisms are introduced: (1) a 'preheater' module that turns cold-start SA queries into input-conditioned queries via a small self-distilled Transformer decoder block, and (2) a differentiated recurrence that uses three SA iterations on the first video frame but only one on subsequent frames. The method is instantiated on top of DIAS for images (SmoothSAi) and RandSF.Q for videos (SmoothSAv). Experiments cover object discovery (ClevrTex, COCO, VOC, MOVi-C/D, YTVIS), object recognition (COCO, YTVIS), and VQA (GQA, CLEVRER), with ablations and a self-distillation robustness stress test. The paper claims new state-of-the-art on both image and video OCL benchmarks and consistent downstream improvements.

Significance. If the stronger claims were accurate, the paper would make a useful contribution to object-centric learning: addressing cold-start queries and recurring-transform homogeneity are plausible design principles, and the two modifications are simple and cheap. The manuscript's strengths include multi-seed experiments across many benchmarks, ablations of the key design choices, released source code/checkpoints/logs, and an explicit discussion of limitations. However, the headline 'new state-of-the-art' and 'consistent performance boosts' claims are not supported by the paper's own tables: DIAS outperforms SmoothSAi on ClevrTex, RandSF.Q outperforms SmoothSAv on MOVi-C ARI, and SmoothSAv strongly regresses in bounding-box IoU on YTVIS. The central empirical claims therefore need to be substantially tempered. The theoretical justification in Section 3.1 is presented as a contraction-based guarantee but is acknowledged in Appendix B to be only illustrative, and the self-distillation bootstrap is a genuine stability risk. With corrected claims and clearer reporting, the paper could still be a solid incremental contribution, but in its current form the conclusions outrun the evidence.

major comments (4)
  1. [§1 (c3), §4.2, Table 1] The contribution c3 claims 'new state-of-the-art on both image and video OCL benchmarks.' This is contradicted by the paper's own Table 1 on ClevrTex: DIAS achieves ARI 80.9, mBO 63.3, mIoU 61.9 while SmoothSAi achieves 76.8, 60.6, 58.9. DIAS is better on all three metrics. Similarly, in Table 2 on MOVi-C, RandSF.Q has ARI 65.4 versus SmoothSAv's 50.9. The claims in Section 4.2 that SmoothSA 'defeats all baselines by a large margin' and achieves 'overall new SotA' are therefore inaccurate. The paper should either restrict the SOTA claim to the benchmarks where it actually holds (e.g., COCO/VOC mIoU, YTVIS mIoU) or carefully qualify the mixed results.
  2. [§4.3, Table 3] Table 3 reports object recognition results, but the bbox IoU regression metric severely regresses on both YTVIS benchmarks: on YTVIS-HQ, SmoothSAv gives bbox IoU 42.6 versus SlotContrast's 51.5; on YTVIS-2022, 39.2 versus 48.2. This is a large drop. The text states 'the object recognition accuracy ... are improved a lot' and c4 claims 'consistent performance boosts on downstream tasks.' Such claims are only valid for classification accuracy, not for the regression (localization) component of recognition. The manuscript needs to acknowledge this tradeoff and temper the 'consistent boosts' language, or provide an explanation for the IoU regression.
  3. [§3.1, Eqs. (3)-(7), Appendix B] The formal motivation assumes the SA aggregation transform Φ_S is a contraction with α<1 (Eq. 3), which is not established for the actual SA module. Appendix B concedes that the Banach fixed-point argument is 'illustrative rather than a rigorous global guarantee.' Moreover, Eq. (5) posits the existence of a preheater that moves queries closer to the fixed point, but the actual preheater is trained by self-distillation against the currently produced slots (Eq. 18). If those slots are poor, the preheater can learn to produce poor queries, the risk the paper itself flags in Section 6. The stress test in Appendix A (1k random corruptions over 100k steps, with no reported corruption magnitude or schedule) is too narrow to establish stability. The theoretical argument should be presented as intuition, and the empirical robustness analysis needs to be strengthened before the paper can claim tha
  4. [§4.5, Table 5] Several ablation numbers in Table 5 appear internally inconsistent with the main results. The rows labeled 'ARI + ARIfg @COCO' report values such as 68.3, 63.3, and 56.9, but Table 1 lists COCO ARI=29.3 and ARIfg=41.3 for SmoothSAi, whose sum is 70.6, not 68.3. In the YTVIS ablation, the 'shared' row for the first/non-first weight-sharing experiment reports 68.3, the same number as the COCO preheater row, while Table 2's YTVIS-HQ ARI+ARIfg for SmoothSAv is 42.4+63.0=105.4. The table lacks a clear definition of the metric and the numbers seem to be duplicated or mislabeled. This makes the ablation conclusions difficult to interpret and needs to be corrected.
minor comments (5)
  1. [§4.2] 'mIoU is the most strick metric' should be 'strict.'
  2. [Figure 2] The figure caption and the illustration contain repeated artifacts such as 'preheatpreheat...' and 'differentiate...', which obscure the intended text. Please replace with a clean figure.
  3. [§4.3] 'firstly convert all images into slots representation' should be 'first convert all images into slot representations' for grammatical clarity.
  4. [Appendix A] The stress test gives no details about the corruption amplitude or schedule; only 'random corruption' is mentioned. Please specify the noise level, the exact timing, and the metric changes in a more transparent form.
  5. [§3.1, Eqs. (13)-(15)] The notation around θ and the unrolling of iterations is hard to follow; some symbols are used without definition. Clarifying the derivation would help readers assess the gradient argument.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claims are supported by external benchmarks and ablations; the acknowledged self-distillation bootstrap and self-citations are not load-bearing reductions.

full rationale

The paper's derivation chain is not circular in the sense targeted by the analysis. The preheater is trained by self-distillation to approximate the SA module's own output slots (Eq. 18, with stop-gradient in Eq. 17). This is an explicitly acknowledged bootstrap: the paper states in Remark 1 that 'good slots lead to better preheated queries, and in turn better preheated queries lead to better slots,' and Section 6 admits that bad slots could lead to worse preheated queries. However, the claimed improvements are not derived from this feedback loop alone; they are empirically validated on held-out external benchmarks (ClevrTex, COCO, VOC, MOVi-C/D, YTVIS, GQA, CLEVRER) against multiple baselines, including non-self methods such as SPOT and SlotContrast. The theoretical motivation in Section 3.1 is a conditional contraction argument: if a preheater P makes queries closer to the fixed point (Eq. 5), then the preheated run is closer after three iterations. This is a tautological conditional, not a claim that the preheater is defined to be such a map; Appendix B explicitly disclaims the Banach fixed-point interpretation as 'illustrative theoretical motivation rather than a rigorous global guarantee' and provides empirical local contraction validation. The choice of three vs one SA iterations is heuristic and validated by ablations (Table 5), not derived from a fitted parameter. The self-citations to the authors' prior DIAS and RandSF.Q appear as building blocks and baselines, but the central empirical results are not reduced to those citations; the method is evaluated on independent benchmarks and compared with externally developed methods. Thus there is no specific equation or fitted parameter that is renamed as a prediction, and no load-bearing self-citation chain. The overbroad 'new state-of-the-art' wording is a correctness/calibration concern, not circularity. Score 1 reflects the presence of self-citations and the acknowledged bootstrap, neither of which is load-bearing.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities, but it depends on several assumptions: the contraction picture of SA (admitted to be illustrative), existence of gradient bounds used in the theoretical motivation, and stability of the self-distillation bootstrap. The hyperparameters (iterations 3+1, preheat loss weight 100) are fitted to the tasks and not derived.

free parameters (3)
  • first_frame_SA_iterations = 3
    Chosen empirically through ablation (Table 5: 3+1 vs 1+1 vs 3+3); the paper admits in Limitations that this is not automatic.
  • non_first_frame_SA_iterations = 1
    Chosen empirically through ablation; paired with first frame count.
  • preheat_loss_weight = 100
    Selected from grid search (10/50/100/200) on COCO (Table 5); 100 times larger than the reconstruction loss.
assumptions (3)
  • domain assumption Slot Attention aggregation is a contraction with a unique fixed point
    Eq (3) in Sec 3.1 cites 'usually a contraction' from Jia et al. 2023; Appendix B admits it is only an illustrative local picture, 'not a rigorous global guarantee.'
  • domain assumption Existence of Lipschitz bounds B and L for SA parameter gradients and loss
    Eq (10)-(12) assume bounds on Jacobians without proof; the subsequent gradient analysis is not rigorous and appears internally inconsistent.
  • ad hoc to paper Self-distillation bootstrap remains stable (the preheater does not amplify slot errors)
    The preheater is trained against SA's own output (bootstrap); the paper itself flags this as a possible limitation in Sec 6 and only provides a limited stress test in Appendix A.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Smoothing Slot Attention Iterations and Recurrences." pith.science (2026). https://pith.science/paper/XZF5JM37

@misc{pith2026250805417,
  author       = {Pith},
  title        = {Pith review of: Smoothing Slot Attention Iterations and Recurrences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XZF5JM37}},
  note         = {Machine review of arXiv:2508.05417}
}
read the original abstract

Slot Attention (SA) lies at the heart of mainstream Object-Centric Learning (OCL). Image features can be aggregated into object-level representations by SA \textit{iteratively} refining cold-start query slots. For video, such aggregation proceeds by SA \textit{recurrently} shared across frames, with queries cold-started on the first frame while transitioned from the previous frame's slots thereafter. However, cold-start queries lack sample-specific cues thus hindering precise aggregation on image or video's first frame; Non-first frames' queries are already sample-specific thus requiring aggregation transforms different from the first frame. We address these issues with our \textit{SmoothSA}: (1) To smooth SA iterations on image or video's first frame, we \textit{preheat} cold-start queries with rich input-feature information, by a tiny module self-distilled inside OCL; (2) To smooth SA recurrences across video's first and non-first frames, we \textit{differentiate} the homogeneous aggregation transforms by using full and single iterations respectively. Comprehensive experiments on object discovery, recognition and visual reasoning validate our method's effectiveness. Further visual analyses illuminate the underline mechanisms. Our \textit{source code}, \textit{model checkpoints} and \textit{training logs} are provided on https://github.com/Genera1Z/SmoothSA.

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Internalizing Temporal Consistency in Video Object-Centric Learning without Explicit Regularization

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    By decomposing slot channels into static and dynamic subspaces via CCD and using CTR for cross-temporal feature reconstruction, the method internalizes temporal consistency without explicit SSC loss.

  2. Cycle Consistency in Video Object-Centric Learning

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Implicit cycle consistency on the reconstruction manifold avoids feature collapse in video object-centric learning unlike explicit slot-space consistency.

  3. Object-Centric Vision Token Pruning for Vision Language Models

    cs.CV 2025-11 reject novelty 6.0 of 10

    OC-VTP uses Slot Attention to select a few 'object-representative' vision tokens, reporting high accuracy retention at extreme pruning ratios, but the claimed guarantee is undercut by a non-differentiable selection an...

Reference graph

Works this paper leans on

18 extracted references · 17 canonical work pages · cited by 3 Pith papers

  1. [1]

    Self-supervised object- centric learning for videos.Advances in Neural Informa- tion Processing Systems, 36:32879–32899, 2023

    Aydemir, G., Xie, W., and Guney, F. Self-supervised object- centric learning for videos.Advances in Neural Informa- tion Processing Systems, 36:32879–32899, 2023

    work page 2023

  2. [2]

    Attention over Learned Object Embeddings Enables Complex Visual Reasoning.Advances in neural informa- tion processing systems, 34:9112–9124, 2021

    Ding, D., Hill, F., Santoro, A., Reynolds, M., and Botvinick, M. Attention over Learned Object Embeddings Enables Complex Visual Reasoning.Advances in neural informa- tion processing systems, 34:9112–9124, 2021

    work page 2021

  3. [3]

    SA Vi++: Towards End-to-End Object-Centric Learning from Real-World Videos.Advances in Neural Informa- tion Processing Systems, 35:28940–28954, 2022

    Elsayed, G., Mahendran, A., Van Steenkiste, S., et al. SA Vi++: Towards End-to-End Object-Centric Learning from Real-World Videos.Advances in Neural Informa- tion Processing Systems, 35:28940–28954, 2022

    work page 2022

  4. [4]

    Improving Object-centric Learning with Query Optimization

    Jia, B., Liu, Y ., and Huang, S. Improving Object-centric Learning with Query Optimization. InThe Eleventh Inter- national Conference on Learning Representations, 2023

    work page 2023

  5. [5]

    Object-Centric Slot Diffusion.Advances in Neural Information Process- ing Systems, 2023

    Jiang, J., Deng, F., Singh, G., and Ahn, S. Object-Centric Slot Diffusion.Advances in Neural Information Process- ing Systems, 2023

    work page 2023

  6. [6]

    Spot: Self-Training with Patch-Order Permu- tation for Object-Centric Learning with Autoregressive Transformers

    Kakogeorgiou, I., Gidaris, S., Karantzalos, K., and Ko- modakis, N. Spot: Self-Training with Patch-Order Permu- tation for Object-Centric Learning with Autoregressive Transformers. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 22776–22786, 2024

    work page 2024

  7. [7]

    Conditional Object-Centric Learning from Video.International Con- ference on Learning Representations, 2022

    Kipf, T., Elsayed, G., Mahendran, A., et al. Conditional Object-Centric Learning from Video.International Con- ference on Learning Representations, 2022

    work page 2022

  8. [8]

    SlotPi: Physics-informed Object-centric Reasoning Models

    Wang, H., Zhang, Y ., Liu, H., Wang, Z., Yu, F., et al. SlotPi: Physics-informed Object-centric Reasoning Mod- els.arXiv preprint arXiv:2506.10778, 2025a

    work page Pith review arXiv

Show all 18 references
  1. [9]

    Metaslot: Break through the fixed number of slots in object-centric learning.arXiv preprint arXiv:2505.20772, 2025

    Liu, H., Zhao, R., Chen, H., and Pajarinen, J. Metaslot: Break through the fixed number of slots in object-centric learning.arXiv preprint arXiv:2505.20772, 2025

    2025

  2. [10]

    Object- Centric Learning with Slot Attention.Advances in Neural Information Processing Systems, 33:11525–11538, 2020

    Locatello, F., Weissenborn, D., Unterthiner, T., et al. Object- Centric Learning with Slot Attention.Advances in Neural Information Processing Systems, 33:11525–11538, 2020

    2020

  3. [11]

    Temporally consistent object-centric learning by contrasting slots

    Zadaianchuk, A. Temporally consistent object-centric learning by contrasting slots. InProceedings of the Com- puter Vision and Pattern Recognition Conference, pp. 5401–5411, 2025

    2025

  4. [12]

    Bridging the Gap to Real-World Object-Centric Learning.Interna- tional Conference on Learning Representations, 2023

    Seitzer, M., Horn, M., Zadaianchuk, A., et al. Bridging the Gap to Real-World Object-Centric Learning.Interna- tional Conference on Learning Representations, 2023

    2023

  5. [13]

    R., Van De Sande, K

    Uijlings, J. R., Van De Sande, K. E., Gevers, T., and Smeul- ders, A. W. Selective Search for Object Recognition. International Journal of Computer Vision, 104:154–171, 2013. Van Den Oord, A., Vinyals, O., and Kavukcuoglu, K. Neural Discrete Representation Learning.Advances in...

    2013

  6. [14]

    N., Kaiser, Ł., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. At- tention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  7. [15]

    and Behnke, S

    Villar-Corrales, A. and Behnke, S. Playslot: Learning inverse latent dynamics for controllable object-centric video prediction and planning. InForty-second Interna- tional Conference on Machine Learning, 2025

    2025

  8. [16]

    Object- centric video prediction via decoupling of object dynam- ics and interactions

    Villar-Corrales, A., Wahdan, I., and Behnke, S. Object- centric video prediction via decoupling of object dynam- ics and interactions. In2023 IEEE International Confer- ence on Image Processing (ICIP), pp. 570–574. IEEE, 2023

    2023

  9. [17]

    Object- Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities.Advances in Neural Infor- mation Processing Systems, 36, 2024

    Zadaianchuk, A., Seitzer, M., and Martius, G. Object- Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities.Advances in Neural Infor- mation Processing Systems, 36, 2024

    2024

  10. [18]

    firewall

    Zhao, R., Li, J., Kannala, J., and Pajarinen, J. Predicting Video Slot Attention Queries from Random Slot-Feature Pairs. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pp. 13208–13216, 2026. 10 Smoothing Slot Attention Iterations and Recurrences A....

    2026

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.