{"id":"a71b8801-9109-44fd-bce3-da9bc817f5bb","arxiv_id":"2501.15878","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An object-centric model that conditions a frozen pretrained diffusion model via slot adapters and an attention-alignment loss, improving unsupervised segmentation and enabling slot-based compositional image editing on COCO.","lead":"SlotAdapt adds trainable adapter layers to a frozen Stable Diffusion model so that slot attention can condition image generation without text embeddings, plus a guidance loss that aligns slot masks with diffusion attention masks. On COCO and VOC it reports better object discovery and reconstruction than prior slot-based methods, and demonstrates compositional editing by swapping or removing slots.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The guidance loss in Eq. 5 aligns a row-softmax slot mask with a column-softmax diffusion mask; exact equality is impossible for unequal numbers of pixels and slots, so the 'mutual alignment' claim is not what the loss actually enforces, and Table 2 is the only support for it.","rationale":"The paper's central claim is that adapter-based slot conditioning plus a mutual attention-guidance loss gives SOTA object discovery and compositional generation. For that claim to hold, the guidance loss must actually align the two masks in the way asserted. Eq. 4-5 are the only formal specification of this mechanism, and they are internally inconsistent: the two masks are normalized along different axes, so the loss cannot have the stated optimum. This is not a disagreement with the community; it is a mathematical property of the defined objective. The paper's own Table 2 is the sole ablation of this component, and it shows the object-discovery metric FG-ARI decreasing under the recommended setting, which is consistent with the loss distorting masks rather than aligning them. Without error bars or a second dataset, the mBO/mIoU improvements cannot be distinguished from noise or from a side effect of the renormalization. The reviewer's conditional verdict is appropriate: the empirical rankings may be correct, but the explanation of the main novel component is not yet credible. My proposed check directly replaces the ill-posed BCE with a well-posed alignment loss and re-runs the same ablation; this settles the mechanism question without requiring retraining from scratch. If the improvements persist, the paper's story is fine; if they vanish, the paper should either correct the loss or revise the claim. Thus I do not move the verdict away from CONDITIONAL.","tokens_in":18460,"tokens_out":14883,"duration_ms":155143,"concrete_test":"Re-run the COCO ablation of Table 2 with both masks normalized on the same axis: replace BCE(ASA, ADM^T) with, e.g., BCE between row-softmax versions of ASA and row-softmax(ADM^T) (or a symmetric KL after matching normalizations), and keep all other settings fixed. If the joint-guidance improvements in mBO/mIoU disappear or reverse, the reported gains are an artifact of the ill-posed BCE rather than genuine slot/cross-attention alignment; if they survive, the normalization mismatch is not the active cause and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. 4 defines ASA as a P×N softmax over slots (each row sums to 1) and ADM as an N×P softmax over pixels (each row sums to 1), so C = ADM^T has each column summing to 1. For C=A, A would have to be both row- and column-stochastic; summing entries gives P on one side and N on the other, which is impossible when P≠N (here P=1024, N=7). Hence BCE in Eq. 5 cannot drive the masks to the transposed instance segmentation the text describes; its optimum is a compromise that respects the two incompatible normalizations. Because BCE treats each entry as an independent Bernoulli probability, it also gives no mechanism for preferring object-like structure. The only evidence for this loss is the COCO ablation in Table 2, which lacks error bars and shows the signature object-discovery metric FG-ARI decreasing under joint guidance (42.3→41.4) while mBO/mIoU rise. That pattern is exactly what one would expect from a loss that renormalizes rather than aligns masks. This is load-bearing because the attention-guidance loss is the paper's main novel contribution and is cited in the conclusion as a key reason for the SOTA results; if the loss is not doing what is claimed, the mechanism behind the gains is unexplained, even if the empirical ranking survives.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SlotAdapt, an object-centric learning method that conditions a frozen pretrained Stable Diffusion decoder on slots produced by a DINOv2/CNN slot-attention encoder. Two additions are proposed: adapter cross-attention layers dedicated to slot conditioning, with a pooled 'register' token routed through the original text cross-attention, and a mutual guidance loss that aligns the slot-attention mask with the adapter cross-attention mask via binary cross-entropy (Eq. 5). The method is evaluated on MOVi-E, VOC, and COCO with FG-ARI, mBO, mIoU, and reconstruction/compositional FID/KID; the authors report gains over LSD and SlotDiffusion, e.g., COCO FG-ARI 41.4–42.3 vs 37.2, instance mBO 35.1 vs 31.0, and reconstruction FID 10.857 vs 19.448, plus qualitative compositional editing on COCO.","tokens_in":18826,"tokens_out":9949,"duration_ms":87941,"significance":"If its claims hold, the paper makes a useful contribution: it shows that adapter-based conditioning can transfer pretrained diffusion generative power to slot-based decoders without text-centric cross-attention, with large FID/KID improvements over both training-from-scratch (SlotDiffusion) and pretrained-but-text-conditioned (LSD) baselines. Strengths include the use of standard benchmarks and metrics, comparisons against both unsupervised and externally supervised concurrent work (GLASS, SPOT), and a reasonably complete set of ablations (conditioning blocks, register token, guidance variants, slot count, CFG scale). The main novel component, the mutual attention guidance loss, is mechanistically underspecified and its ablation evidence is mixed (Table 2): the explanation of why it helps is not supported by the mathematics in Eqs. (4)–(5). Because the no-guidance variant already outperforms the baselines on most metrics, the paper's headline empirical ranking is plausibly robust to a reframing of the guidance mechanism, but the loss's role needs to be restated and better evidenced.","major_comments":[{"comment":"The claimed mechanism of the mutual guidance loss is not compatible with the definitions in Eqs. (4)–(5). ASA is a P×N row-softmax (each pixel row sums to 1, total mass P) and ADM is an N×P row-softmax (each slot row sums to 1, so ADM^T has column sums of 1, total mass N). For P=1024 and N=7, exact equality ASA = ADM^T is impossible because the total mass would have to be both P and N; the text's statement that ADM 'should converge to the transpose of the instance segmentation mask' cannot hold in any exact sense, and the BCE objective in Eq. (5) instead settles at a compromise between two incompatible normalizations. This matters because the guidance loss is the paper's main novel component and is credited in the Conclusion with the SOTA results. The Table 2 ablation is consistent with the loss acting as a renormalization rather than an object-level alignment: joint guidance raises mBOi (31.5 to 35.1) and mBOc (34.8 to 39.2) but lowers FG-ARI (42.3 to 41.4). Please (i) explicitly acknowledge the normalization mismatch and reframe the loss as a soft, regularizing alignment, or modify the normalization (e.g., row-normalize both masks over pixels, or use a cost that accounts for the differing marginals); (ii) report a direct measure of agreement between ASA and ADM^T over training to substantiate 'alignment'; and (iii) discuss the FG-ARI decrease under joint guidance.","section":"§3.2, Eqs. (4)–(5)"},{"comment":"All quantitative results in the paper are single runs with no error bars and no stated number of seeds. This is load-bearing for the guidance claim because the Table 2 differences are small in absolute terms (FG-ARI 42.3 vs 41.4 vs 43.3; mBOi 31.5 vs 35.1) and slot-attention training is known to have run-to-run variance of this order. Please report mean ± std over at least three seeds for the guidance ablations and the main VOC/COCO comparisons, and state the seed count; if the guidance gains are within noise on some metrics, say so explicitly.","section":"§4, Tables 2 and 4"},{"comment":"The compositional generation results, which support the headline claim of 'first successful compositional generation on COCO', are not reproducible as reported: the text gives no protocol for how the right half of Table 3 was computed (which slot manipulations were used, how many images, which reference set, sampling steps, CFG value). Please add a full experimental protocol for the compositional FID/KID evaluation, including the exact editing operations and sample counts, or clearly label these numbers as qualitative-illustration-level.","section":"§4, Table 3"}],"minor_comments":[{"comment":"The Introduction claims 'significantly reduced computational requirements' relative to GLASS, but no runtime, parameter-count, or memory comparison appears anywhere in the paper or appendix; please quantify or remove this claim.","section":"§1 and Table 11"},{"comment":"The 'Slot Average Token' row reports FG-ARI 42.3, mBOi 31.5, and mBOc 34.8, which match the 'No guidance' configuration of Table 2 rather than the joint-guidance main model (41.4, 35.1, 39.2); please clarify which configuration this row represents.","section":"Appendix A.2, Table 9"},{"comment":"Please state explicitly that the slot-attention and adapter cross-attention mechanisms operate at different resolutions and feature dimensions, and that each softmax in Eq. (4) is scaled by its own query/key dimension; as written the shared √D is misleading.","section":"§3.2, Eq. (4)"},{"comment":"There are several typos, including 'arhictectural' at the start of A.1, 'and and our approach' in the same section, and inconsistently spaced 'V AE' throughout; a final proofread is needed.","section":"Appendix A.1"},{"comment":"In the VOC panel, GLASS reports mBOc 36.1 while GLASS† reports 68.4; given the caption's description of the two variants this gap is confusing, and the exact supervision used by each variant (and why the class-level overlap differs so strongly) should be stated.","section":"Appendix A.4, Table 11"},{"comment":"The statement that FG-ARI 'does not account for object masks larger than the ground-truth' is unclear; the ARI is computed over the full clustering and would change with mask sizes, so please state precisely which limitation is intended.","section":"§4, Metrics paragraph"},{"comment":"Only a single guidance weight (λ=0.025 with a 40K-iteration warm-up) is reported; a short sensitivity check for λ would strengthen the Table 2 conclusion that joint guidance 'significantly improves' over no guidance.","section":"Appendix A.1, Guidance loss schedule"},{"comment":"No code release is mentioned; given the many implementation details in Appendix A.1, providing training and evaluation code would substantially aid reproducibility of the compositional experiments.","section":"Overall reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The manuscript header states it is already published as an ICLR 2025 conference paper; my report treats the arXiv v3 text as the candidate manuscript and focuses on the technical content as written. The main risk is that the mutual guidance loss, the most novel component, is currently explained by a mechanism (exact transpose alignment) that the math in Eqs. (4)–(5) rules out, and its only dedicated evidence is a single-run ablation with mixed metric directions. The empirical ranking of the overall method is likely robust because the no-guidance variant already beats the baselines on most metrics, so I see this as repairable rather than fatal. The authors should also double-check the 'first compositional generation on COCO' claim against any other concurrent work besides GLASS and the BDD100K-based method they cite."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SlotAdapt is a solid empirical paper. The genuinely new thing is the recipe: freeze a pretrained Stable Diffusion, insert T2I-Adapter-style cross-attention layers for slot conditioning, add a register token pooled from slots, and train with a guidance loss that tries to align slot-attention masks with adapter cross-attention masks. The result is the first slot-based method to do compositional generation/editing on COCO, and the numbers are good: on COCO, SlotAdapt beats SlotDiffusion by about 4 FG-ARI and 4 instance mBO, and reconstruction FID drops from 19.4 to 10.9. The ablations are genuinely useful—conditioning blocks, register token, guidance variants, slot count, CFG scale—and the appendix gives fair comparisons with GLASS and SPOT, including the caveat that GLASS uses extra supervision.\n\nThe soft spot is the guidance loss. The stress-test note is right: ASA is P×N row-stochastic, ADM is N×P row-stochastic, so BCE cannot drive them to exact transposes when P≠N. The text's claim that ADM should converge to the transpose of the instance segmentation mask is therefore wrong on its face. The loss is also self-referential—both masks come from the same slots—so it could just push both toward a shared compromise. Table 2 shows exactly that pattern: joint guidance lowers FG-ARI (42.3→41.4) while raising mBO/mIoU. That may be a fine trade-off, but the paper doesn't explain why, and it's the one place where the mechanism is unclear. This is not fatal: the empirical gains over baselines are large and the method is well-engineered. But it needs a better explanation, perhaps with gradient analysis or a toy example, and error bars.\n\nOther issues: no code, single runs, and the CFG scale is selected on the validation FID. These are minor but should be fixed in a revision.\n\nWho gets value: people working on object-centric learning with diffusion decoders, or on adapting frozen text-to-image models to non-text conditioning. It deserves a serious referee, and with code and error bars it would be a useful reference point. I would send it to peer review.","headline":"A clean, well-ablated empirical contribution that delivers first slot-based compositional generation on COCO; the mutual guidance loss is the weakest link but not a fatal one.","tokens_in":19319,"tokens_out":3880,"would_cite":true,"duration_ms":36315,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SlotAdapt conditions a frozen pretrained diffusion model on slots through adapter layers and a mutual attention guidance loss, reporting state-of-the-art unsupervised object discovery, generation, and compositional editing on COCO.","keywords":["object-centric learning","slot attention","diffusion models","adapter layers","compositional generation","unsupervised segmentation","attention guidance","Stable Diffusion"],"falsifier":"Train SlotAdapt on COCO after breaking the alignment, for example by shuffling slot identities in the guidance loss, and check whether segmentation and generation metrics drop; if the metrics barely change, or if $A^{SA}$ and $A^{DM}$ align by collapsing to the same background-dominated map, the mutual guidance loss is not doing the object-alignment work claimed.","tokens_in":18300,"feed_emoji":"🧩","tokens_out":11597,"duration_ms":95411,"temperature":0.7,"pith_summary":"SlotAdapt tries to fix a mismatch in object-centric learning: slot-based models that reuse pretrained diffusion decoders inherit a text-centric conditioning bias, while models that retrain a diffusion decoder from scratch lose its generative power. The paper's solution keeps Stable Diffusion frozen and inserts dedicated adapter cross-attention layers that accept slot vectors, plus a register token formed by pooling the slots to carry global scene context through the original text cross-attention. A mutual attention guidance loss applies binary cross-entropy between the slot attention mask and the adapter cross-attention mask, pulling both toward object-level masks without external supervision. On MOVi-E, VOC, and COCO the method reports improved segmentation and generation over LSD, SlotDiffusion, DINOSAUR, and related baselines, with COCO reconstruction FID of 10.857 versus 19.448 for SlotDiffusion. It also reports the first compositional generation and slot-editing results on COCO, arguing that unsupervised object-centric learning can ride on the scale and visual quality of pretrained diffusion models.","feed_headline":"SlotAdapt beats prior object-centric models on COCO unsupervised","feed_subtitle":"Adapter layers and a mutual attention loss let a frozen diffusion decoder discover objects and edit scenes.","key_machinery":"The load-bearing machinery is the adapter cross-attention layer: an extra cross-attention block inserted after every existing cross-attention layer in all downsampling and upsampling blocks of the frozen Stable Diffusion UNet, taking the extracted slots as conditioning input instead of text embeddings. A register token, computed by mean-pooling the slots (or, in an ablation, the image features), is fed to the original text cross-attention layers to absorb background and global context. The mutual attention guidance loss is $L_{\\mathrm{guidance}} = \\mathrm{BCE}(A^{SA}, A^{DM\\top})$, where $A^{SA}$ is the slot attention mask and $A^{DM}$ is the adapter cross-attention mask averaged over heads; the paper finds that joint guidance (no gradient stopping) works best, and applies the guidance only in the third upsampling block, where masks align best with objects. The full objective is $L = L_{\\theta} + \\lambda L_{\\mathrm{guidance}}$, with $\\lambda$ ramped from 0 to 0.025 after 40K iterations.","core_discovery":"SlotAdapt's central claim is that a frozen pretrained Stable Diffusion model can serve as an object-centric decoder when conditioned through dedicated adapter cross-attention layers rather than through its text-trained cross-attention. Slots extracted by slot attention from visual backbone features are fed to adapters inserted after every cross-attention layer in the UNet, while a register token formed by mean-pooling the slots is fed to the original text cross-attention to absorb global scene context. A mutual guidance loss applies binary cross-entropy between the slot attention mask and the transposed adapter cross-attention mask (averaged over heads), trained jointly without gradient stopping, so that both masks converge toward object-level segmentation. The paper reports that this configuration outperforms prior slot-based diffusion and self-supervised baselines in object discovery and generation on MOVi-E, VOC, and COCO, and presents the first compositional generation and slot-editing results (object removal, replacement, addition) on COCO, all without external supervision.","pith_inferences":["A next step beyond the paper is to learn a mapping from slots to text embeddings so compositional edits could be requested by prompt rather than by manually selecting slots.","The adapter-plus-guidance recipe may transfer to other non-text conditioning signals for frozen diffusion models, using cross-attention masks as a general self-supervisory alignment signal.","Because the paper reports that 80 slots degrade performance relative to 7, slot merging and splitting, acknowledged as a limitation, would test whether the mask-alignment objective scales to denser scenes.","The mutual mask-alignment loss could also be evaluated as a binding objective for slot-based video or world models, where temporal consistency of attention masks might replace image reconstruction as the training signal."],"forward_implications":["A frozen pretrained diffusion decoder can be repurposed for slot conditioning without retraining its weights, so object-centric models keep the full generative prior.","Because slots no longer need to live in text embedding space, they can bind to object instances rather than semantic classes, which the reported instance-level mBO gains on COCO support.","The register token gives global context a dedicated pathway, allowing slots to focus on distinct objects; the paper's ablations show consistent gains from adding it.","Joint mask guidance improves object-level alignment and reduces part-whole fragmentation, and the gains transfer to generation: COCO reconstruction FID drops from 19.448 (SlotDiffusion) to 10.857.","Slots become an editing interface: removing, replacing, or adding slots produces coherent compositional edits on real COCO images, which the paper reports as the first such results."],"supporting_citations":[{"why":"Supplies the slot attention mechanism that produces the object-centric slot representations used throughout SlotAdapt.","marker":"Locatello et al. (2020)"},{"why":"Supplies the T2I-Adapter design of extra cross-attention layers that SlotAdapt adapts for slot conditioning of the frozen diffusion model.","marker":"Mou et al. (2024)"},{"why":"Supplies the pretrained Stable Diffusion latent diffusion model used as the frozen slot decoder.","marker":"Rombach et al. (2022)"},{"why":"LSD is the main pretrained-diffusion baseline and the slot-conditioned diffusion approach that SlotAdapt extends and compares against.","marker":"Jiang et al. (2023)"},{"why":"SlotDiffusion is the train-from-scratch slot-conditioned diffusion baseline and the strongest prior for real-world generation.","marker":"Wu et al. (2023b)"},{"why":"Supplies the register token concept used to give the diffusion model a dedicated global-context pathway.","marker":"Darcet et al. (2024)"},{"why":"GLASS is the concurrent work that also aligns cross-attention masks with slot attention but requires external captions or class labels; SlotAdapt positions against it.","marker":"Singh et al. (2024)"},{"why":"DINOSAUR is a leading fully unsupervised baseline and the source of the real-world evaluation protocol and training splits.","marker":"Seitzer et al. (2023)"}],"fun_headline_variants":["SlotAdapt: frozen diffusion, slot attention, composition","Adapters turn frozen diffusion into object-centric engine","Object discovery + compositional generation via SlotAdapt","SlotAdapt: first COCO slot-editing without supervision"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that binary cross-entropy between the slot attention mask and the transposed adapter cross-attention mask, trained jointly without gradient stopping, drives both masks toward genuine object-level structure instead of a trivial or degenerate agreement.","fun_headline_variants_meta":{"raw":{"variants":["SlotAdapt: frozen diffusion, slot attention, composition","Adapters turn frozen diffusion into object-centric engine","Object discovery + compositional generation via SlotAdapt","SlotAdapt: first COCO slot-editing without supervision"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000193,"raw_usage":{"total_tokens":1318,"prompt_tokens":881,"completion_tokens":437,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":376}},"tokens_in":497,"tokens_out":437,"duration_ms":4919,"temperature":1.0,"reasoning_tokens":376,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:49:55.986849+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SlotAdapt on COCO after breaking the alignment, for example by shuffling slot identities in the guidance loss, and check whether segmentation and generation metrics drop; if the metrics barely change, or if $A^{SA}$ and $A^{DM}$ align by collapsing to the same background-dominated map, the mutual guidance loss is not doing the object-alignment work claimed.","supporting_citations":[],"review_version":1}