{"id":"f4bb1a3f-4902-4fdd-b126-3b64312a33d3","arxiv_id":"2504.19506","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SynergyAmodal co-synthesizes a 16K amodal dataset from EntitySeg images with human and model guidance, and trains a diffusion model that completes occluded objects with optional text control.","lead":"The paper introduces SynergyAmodal, a pipeline that generates a 16,000-pair amodal completion dataset by combining web images, human filtering, and generative diffusion models, then uses it to train a text-controllable deocclusion model called DeoccAnything. A generalist reader might care because the method reports state-of-the-art amodal completion results and adds open-vocabulary text control to a task previously limited to fixed categories.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim rests on unvalidated pseudo-labels: SynergyAmodal16K masks are produced by ZIM on SD3-refined partial-model outputs, with no independent human amodal check to rule out self-training bias.","rationale":"The reader's weakest assumption is exactly the pseudo-label validity assumption, and I agree it is load-bearing. The reported benchmark numbers are the only evidence offered for label quality, but they are confounded: the full model is initialized from SD2 inpainting, the labels come from SD3 plus ZIM, and mIoU is computed with best-of-8 selection. A direct human-annotation check on a random subset would isolate dataset validity. Because the preprint does not release the dataset or this validation, conditional acceptance is appropriate; my stress test does not change that verdict.","tokens_in":18098,"tokens_out":9456,"duration_ms":96292,"concrete_test":"Select 200 random SynergyAmodal16K samples; show each original modal image (without the synthesized completion) to independent annotators and ask them to draw the complete amodal mask of the target instance; compute mIoU between these human masks and the ZIM-derived pseudo-masks. If the IoU is at or above the inter-annotator agreement on COCOA (typically >85), the labels are valid; if it is below roughly 75, the pseudo-label assumption fails and the central claim needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that SynergyAmodal16K is a 'high-quality paired amodal dataset' sufficient to train a state-of-the-art deocclusion model—depends on treating the co-synthesis outputs in Sec. 3.3 as valid amodal ground truth. In that pipeline, the partial completion model proposes deocclusion results; Stable Diffusion 3 refines them with noise strengths 0.5/0.75/1; human annotators select aesthetically plausible variants; and ZIM generates the fine-grained amodal masks from the refined image. The amodal shapes used as supervision are therefore not independently annotated: they are traced by a zero-shot matting model from images produced by the same family of diffusion priors used to initialize the full completion model. The paper reports no quantitative validation of these pseudo-labels against human amodal annotations (no subset IoU, no inter-annotator agreement), and Appendix D only shows examples and statistics. Without such validation, the high COCOA/BSDSA mIoU could reflect the diffusion prior's inductive bias plus best-of-8 oracle selection rather than genuine amodal understanding, leaving the data-human-model contribution unsubstantiated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SynergyAmodal, a framework for co-synthesizing a large-scale amodal completion dataset (SynergyAmodal16K, about 16K pairs) from in-the-wild images via a three-stage pipeline: (1) an order-aware self-supervised partial completion model trained on EntitySeg with synthetically added occluders; (2) a data-human-model co-synthesis stage in which the partial model's deocclusion outputs are filtered and selected by human experts, refined with Stable Diffusion 3, and annotated by ZIM and InternVL; and (3) a full text-conditioned diffusion completion model trained on the synthesized pseudo-labels. The paper reports state-of-the-art mIoU and FID on COCOA and BSDSA, demonstrates textual controllability qualitatively, and releases code, dataset, and models.","tokens_in":18372,"tokens_out":8550,"duration_ms":77066,"significance":"The paper addresses a practical bottleneck in amodal completion, and the 'data-human-model' co-synthesis idea is timely and potentially impactful. The external evaluation on human-annotated COCOA and BSDSA masks is an appropriate way to test generalization, and the proposed order-grounded self-supervised algorithm is a concrete improvement over the SSSD dual-occlusion ambiguity. The promise to release code, dataset, and models is valuable for the community. However, the significance of the reported results is currently limited by insufficient validation of the synthesized pseudo-labels and by an evaluation protocol that favors the proposed stochastic generative method.","major_comments":[{"comment":"The amodal masks of SynergyAmodal16K are generated by ZIM from Stable Diffusion 3-refined outputs of the partial completion model, and the human annotators filter and select candidates but do not independently verify the fine-grained amodal masks; Appendix D provides only example images and aggregate statistics, with no subset IoU or inter-annotator agreement against human-annotated amodal masks. Because the full completion model is trained directly on these pseudo-labels, the claim that SynergyAmodal16K is 'high-quality' and capable of training a state-of-the-art deocclusion model is not yet substantiated, and the risk of self-training on the diffusion prior's bias remains open. Please add a quantitative validation of the pseudo-labels (e.g., mask IoU on a human-re-annotated random subset) or an ablation showing that a model trained on a small human-verified subset performs comparably.","section":"Section 3.3"},{"comment":"The evaluation protocol uses best-of-8 oracle selection for Pix2Gestalt and Ours while SDAmodal and SSSD are evaluated as deterministic methods, which biases the comparison in favor of the stochastic generative methods; the assertion that Ours already outperforms SDAmodal with two or four variations refers to Fig. 5, but Fig. 5 lacks error bars or significance tests and the underlying numbers are not reported, so the claim cannot be checked from the manuscript. Please report mean and standard deviation across seeds or bootstrap confidence intervals, apply a common selection rule to all methods (e.g., best-of-k for every stochastic baseline, or average over samples), and provide significance tests for the headline mIoU differences.","section":"Section 4.2, Table 1 and Fig. 5"},{"comment":"The manuscript does not specify whether FID is computed on the best-of-8 outputs selected for mIoU, on all generated samples, or on a single random sample, and the reference set of 'real unoccluded instances in the dataset' is not defined (which images, how many, what resolution and cropping). This makes the reported FID differences difficult to interpret, especially because FID is sensitive to the number and preprocessing of samples; please specify the complete FID computation (feature extractor, number of samples, preprocessing) and apply it identically to all methods.","section":"Section 4.2 FID paragraph"}],"minor_comments":[{"comment":"The statement that SDAmodal is directly trained on the COCOA dataset appears only after Fig. 5 and is absent from Table 1's caption, so readers may misinterpret the COCOA comparison as a fair zero-shot evaluation; please move this caveat to the main text or table caption.","section":"Section 4.2"},{"comment":"The heading 'Pesudo Data Co-Synthesis' contains a typo; it should be 'Pseudo Data Co-Synthesis'.","section":"Section 3.3"},{"comment":"The limitations section discusses text and shadows but does not mention the potential bias introduced by the pseudo-label generation process; a brief discussion of the dataset-validity limitation would be appropriate.","section":"Section 4.6"},{"comment":"Textual controllability is only demonstrated qualitatively (Fig. 8); adding a quantitative measure (e.g., CLIP similarity between prompt and output, or a small user study) would strengthen the claim of text-driven control.","section":"Section 4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem and the dataset/model could be useful if released. My main hesitation is the lack of pseudo-label validation and the best-of-8 oracle evaluation, which together make it difficult to judge the true magnitude of the reported improvements over SDAmodal. I would not reject the paper; I would require the above issues to be addressed before acceptance. The authors should also be encouraged to state more carefully what 'zero-shot' means given that the pre-trained diffusion priors may have seen COCOA/BSDSA images during their original training."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Best to know: this is a useful systems paper, not a breakthrough. It gives the field a text-controllable open-world amodal completion model and a 16K dataset, with a plausible tripartite pipeline, but the headline SOTA numbers need artifact release and a couple of controlled evaluations before I take them at face value.\n\nWhat is actually new: the order-grounded self-supervised extension fixes a real dual-occlusion ambiguity in SSSD by using InstaOrder annotations rather than random occluders; the human-model co-synthesis loop (filter, refine with SD3, human selection, ZIM/InternVL annotation) is a reasonable way to bootstrap amodal pairs from in-the-wild images; and DeoccAnything does support text-conditioned completion, which earlier deocclusion models don't. The paper also gets credit for a non-cherry-picked qualitative comparison, including failure cases, and for testing on two external benchmarks with human-annotated masks.\n\nSoft spots, in rough order of importance. First, the pseudo-labels are not validated against human amodal masks. The full completion model is trained on ZIM masks taken from SD3-refined outputs of the partial completion model, so a self-training bias is possible. The stress-test note is right that this is unmeasured; I'd only soften the conclusion that it invalidates the central claim. The strong external IoU results suggest the bias isn't catastrophic, but the dataset's \"high-quality\" claim remains an assumption until someone checks a random subset against human annotation. Second, the best-of-8 mIoU protocol inflates the headline for generative methods; they should report mean/median or at least a single-sample number. The claim that the method beats deterministic baselines with 2-4 samples (Fig. 5) helps, but no error bars or significance tests appear. Third, SDAmodal is directly trained on COCOA, so the comparison there is partly apples-to-oranges, though the authors acknowledge it. Fourth, the promised code, dataset, and models are not available, so nothing can be independently verified. The math is straightforward and the citation pattern covers the relevant baselines cleanly.\n\nWho is this for: anyone working on amodal completion, image editing, or synthetic data generation for vision. I would send it to peer review; it deserves referee time. I would cite the dataset if it actually ships and clears the pseudo-label sanity check.","headline":"Useful data-plus-model package; trust the SOTA numbers only after artifacts are out and the pseudo-label loop is sanity-checked against human annotation.","tokens_in":18907,"tokens_out":4166,"would_cite":true,"duration_ms":45636,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a 16K-image co-synthesized dataset of modal-to-amodal pairs can train a single text-conditioned diffusion model that deoccludes arbitrary objects in open-world images.","keywords":["amodal completion","deocclusion","diffusion model","text control","dataset co-synthesis","self-supervised learning","zero-shot generalization","RGBA generation"],"falsifier":"Compute the mIoU of DeoccAnything using exactly one generated sample per instance instead of the best of eight; if single-sample mIoU on COCOA drops to within a point of SSSD's score, the reported advantage comes from sampling selection, not from the synthesized dataset. Independently, retrain the full completion model on the same 16K pairs without the human filtering and Stable Diffusion 3 refinement, using raw pseudo-labels, and check whether zero-shot mIoU and FID fall; if they do not, the co-synthesis stage is not the cause of the gains.","tokens_in":17920,"feed_emoji":"🎨","tokens_out":6631,"duration_ms":61769,"temperature":0.7,"pith_summary":"The paper tries to show that the lack of high-quality amodal ground truth can be overcome by co-synthesizing it: combine in-the-wild modal images for diversity, human filtering for plausibility, and strong generative models for fidelity to automatically produce about 16K paired modal-to-amodal examples. On top of this dataset, the authors train a text-conditioned diffusion model, DeoccAnything, that they claim generalizes zero-shot to open-world occlusion scenarios and lets users steer the hallucinated content with prompts. The quantitative claim is that it reaches 90.3 mIoU and 9.5 FID on COCOA and 90.2 mIoU and 34.3 FID on BSDSA, the best among prior self-supervised, supervised, and model-driven baselines. If true, the work would show that a relatively small, carefully synthesized dataset can replace expensive manual amodal annotation and still yield a controllable, general-purpose deocclusion model.","feed_headline":"16K co-synthesized pairs train a top-scoring deoccluder","feed_subtitle":"A data-human-model pipeline yields a diffusion model that leads COCOA and BSDSA with text-guided amodal completion.","key_machinery":"The load-bearing mechanism is the co-synthesis pipeline built on an order-aware self-supervised partial completion model. The partial model removes occluders one at a time in a learned order, using occlusion order annotations, and is trained so that newly synthesized occluders never cover an existing occluder, avoiding the dual-occlusion ambiguity that corrupted the SSSD baseline; this model produces initial deocclusion results. Those results are then filtered by human experts, refined by Stable Diffusion 3 at multiple noise strengths, selected again by humans, and annotated by ZIM for fine masks and InternVL for captions to yield SynergyAmodal16K. The final DeoccAnything model is a latent diffusion inpainting network that takes the visible image, the modal mask, an optional text prompt, and the full image as a conditioning signal, with a two-stage global-to-local inference strategy that sharpens high-resolution outputs. The order-aware training and the human-in-the-loop co-synthesis are what the claims of diversity, plausibility, and fidelity rest on.","core_discovery":"SynergyAmodal establishes that a tripartite data-human-model collaboration can synthesize an amodal dataset whose quality is high enough to train a full completion diffusion model that performs zero-shot amodal completion and supports open-world text control. The argument has three linked steps: an order-aware self-supervised partial completion model trained on EntitySeg provides plausible initial deocclusions; a filter-refine-select-annotate pipeline, using Stable Diffusion 3 refinement and ZIM and InternVL annotations, turns those outputs into 16K high-quality modal-to-amodal training pairs; and the full completion model, a Stable Diffusion 2 Inpainting architecture with an extra full-image condition and text conditioning, outperforms prior methods on COCOA and BSDSA on both mask mIoU and FID. The paper claims this demonstrates both zero-shot generalization and textual controllability for open-world amodal completion.","pith_inferences":["If the central claim holds, the bottleneck for further scaling is human screening, since about 200 annotator-hours produced only 16K pairs, so the natural next step is automating plausibility filtering with a learned reward model rather than adding more annotators.","The evaluation protocol of reporting best-of-8 samples for generative methods favors methods that produce diverse candidates, so reporting single-sample mIoU would measure the model's average competence instead of its best guess, and future comparisons should include it.","The text-conditioning channel implies a new test for amodal completion: whether the completed appearance is semantically consistent with the prompt, which could be quantified by measuring how often generated contents match the caption semantics.","The same data-human-model loop could transfer to video amodal completion, where temporal occlusion order provides a natural supervision signal analogous to the static occlusion order used here."],"forward_implications":["The released SynergyAmodal16K dataset and DeoccAnything model provide a reusable resource, so other amodal, segmentation, or image-editing methods can be trained or evaluated on the same pairs.","The order-aware self-supervised learning recipe can be applied to any modal segmentation dataset, resolving the dual-occlusion ambiguity that limited prior self-supervised deocclusion.","Because the full model conditions on text, users can control the amodal content for occluded regions, such as the species of an occluded animal or the clothing of a person, a capability prior amodal completion methods did not offer.","Combining the model with SAM enables point-prompted deocclusion, and plugging its RGBA output into image-to-3D pipelines produces complete 3D geometry from occluded single views.","The two-stage global-to-local inference improves output sharpness for large inputs and can be adopted by other latent diffusion inpainting models."],"supporting_citations":[{"why":"Supplies the in-the-wild modal image data with pixel-accurate category-agnostic masks used for both self-supervised training and as the source of SynergyAmodal16K.","marker":"[46]"},{"why":"The self-supervised scene de-occlusion method the paper builds on; its dual-occlusion ambiguity motivates the order-aware training, and it is a main comparison baseline.","marker":"[64]"},{"why":"Pix2Gestalt is the prior generative amodal completion approach whose SA-1B-based dataset lacks physical plausibility, serving as the key comparison and motivation.","marker":"[42]"},{"why":"Stable Diffusion 2 Inpainting is the base latent diffusion model repurposed for both the partial and full completion models and is also used as the RGB-generating baseline.","marker":"[50]"},{"why":"Stable Diffusion 3 is used in the co-synthesis pipeline to refine the initial deocclusion results across multiple noise strengths.","marker":"[12]"},{"why":"Provides the instance-wise occlusion and depth orders that ground the order-aware self-supervised learning.","marker":"[28]"},{"why":"ZIM is used to automatically generate fine-grained amodal masks as part of the annotation stage of the co-synthesis pipeline.","marker":"[23]"},{"why":"InternVL produces the descriptive captions that condition the full completion model and supply the text-control signal.","marker":"[8]"},{"why":"COCOA is the benchmark with expert-annotated amodal masks used for evaluation, and it is also the dataset that the SDAmodal baseline is trained on.","marker":"[70]"},{"why":"BSDSA is the second evaluation benchmark used to test cross-domain zero-shot generalization and appearance fidelity.","marker":"[38]"}],"fun_headline_variants":["Amodal deocclusion with text control via human-AI data synthesis","16K co-synthesized pairs train a text-controlled deoccluder","SynergyAmodal: human-AI pipeline for zero-shot amodal completion","Human-AI co-synthesis unlocks zero-shot text-guided deocclusion","Text-controlled deocclusion from 16K human-refined training pairs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole training rests on the assumption that the pseudo-labels produced by filtering, refining, and annotating the self-supervised model's outputs are accurate and unbiased amodal ground truth, so the model's good test scores reflect true amodal understanding rather than the same diffusion priors that generated its training data.","fun_headline_variants_meta":{"raw":{"variants":["Amodal deocclusion with text control via human-AI data synthesis","16K co-synthesized pairs train a text-controlled deoccluder","SynergyAmodal: human-AI pipeline for zero-shot amodal completion","Human-AI co-synthesis unlocks zero-shot text-guided deocclusion","Text-controlled deocclusion from 16K human-refined training pairs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000689,"raw_usage":{"total_tokens":3154,"prompt_tokens":1007,"completion_tokens":2147,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":2049}},"tokens_in":623,"tokens_out":2147,"duration_ms":14076,"temperature":1.0,"reasoning_tokens":2049,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:51:00.473502+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the mIoU of DeoccAnything using exactly one generated sample per instance instead of the best of eight; if single-sample mIoU on COCOA drops to within a point of SSSD's score, the reported advantage comes from sampling selection, not from the synthesized dataset. Independently, retrain the full completion model on the same 16K pairs without the human filtering and Stable Diffusion 3 refinement, using raw pseudo-labels, and check whether zero-shot mIoU and FID fall; if they do not, the co-synthesis stage is not the cause of the gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the instance-wise occlusion and depth orders that ground the order-aware self-supervised learning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"BSDSA is the second evaluation benchmark used to test cross-domain zero-shot generalization and appearance fidelity."}],"review_version":1}