{"id":"3c70ecf6-ada4-4c2f-bf15-18e4b98c5499","arxiv_id":"2608.00588","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"InstancePin adds per-instance coordinate tokens and mask-guided fusion to a frozen layout-to-image diffusion model, improving FID and mIoU on Cityscapes while reducing visual blending of nearby same-category objects.","lead":"InstancePin is a diffusion-based image generator that adds a coordinate anchor for every object instance in a street-scene layout, so nearby pedestrians or cars are generated separately instead of blurring together. On Cityscapes it reports better FID and mIoU than category-level baselines, with the clearest gains in crowded pedestrian areas.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"InstancePin uses ground-truth instance maps as input that baselines lack; Table 1's gains may reflect extra conditioning, not coordinate pinning.","rationale":"The reader's verdict was CONDITIONAL with the weakest assumption being that FID/mIoU differences are stable and that category-level mIoU captures instance-level disentanglement. I agree with the general thinness of evidence, but I identify a more specific, more load-bearing flaw: the comparison against category-level baselines is not apples-to-apples because InstancePin additionally conditions on ground-truth instance centers P and the binary instance mask B, which baselines never receive. This makes the headline quantitative improvements non-diagnostic: they could stem from the extra instance-level prior alone. The paper itself notes the need for an instance map in Section 3.1, and Section 4.2 gives no indication that baselines were given equivalent instance information. The ablations do not resolve this because every variant in Table 2 uses P and B. The paper also lacks an instance-level evaluation metric, as the reader noted, but the input confound is prior: even if an instance metric were added, the unfair conditioning would remain. I recommend keeping the CONDITIONAL verdict, with the explicit condition that the authors add an instance-conditioned baseline (or remove instance input from InstancePin) and report the result; hence 'UNCHANGED' relative to the reader's verdict.","tokens_in":9093,"tokens_out":9643,"duration_ms":119598,"concrete_test":"Train an instance-conditioned baseline: augment FreestyleNet (or ControlNet) with the same inputs used by InstancePin — concatenate the binary instance mask B to the semantic layout and supply the center set P as an additional control (e.g., a few coordinate tokens or an extra input channel) — using identical data, resolution, and compute budget. If this baseline matches or beats InstancePin's FID/mIoU (37.6/75.1 on Cityscapes), the Table 1 gains do not demonstrate coordinate pinning. Equivalently, run InstancePin with M replaced by a predicted instance map (e.g., from a segmenter) so only S is available; if the advantage over FreestyleNet largely disappears, the model depends on ground-truth instance input.","verdict_should_be":"UNCHANGED","load_bearing_attack":"InstancePin's experimental comparison is confounded by an input asymmetry. Section 3.1 defines M as a given instance map and derives all instance signals (centers P in Eq. 2, binary mask B in Eq. 5) from it; these are used at training and inference. The baselines in Table 1 (OASIS, SDM, ControlNet, Uni-ControlNet, FreestyleNet) are conditioned only on the semantic layout S and text prompt; none receives M, P, or B. The 2.7-point FID and 1.8-point mIoU gains over FreestyleNet are therefore attributable, at least in principle, to the extra instance-level information rather than to the proposed coordinate-pinning attention. The ablations (Table 2) hold M fixed, so they cannot separate the value of the extra input from the value of the architecture. Without an instance-conditioned baseline — e.g., FreestyleNet or ControlNet fed the same instance mask/centers — the central claim that coordinate pinning itself improves instance-addressable generation is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes InstancePin, a layout-to-image diffusion method that adds instance-level control through coordinate pinning. Each instance is represented by a coordinate token derived from its center via Fourier features and a mapping network. An instance-aware adapter runs in parallel with a frozen pretrained backbone, and coordinate pinning attention lets latent positions query these tokens, with the attention maps supervised by instance regions (Eq. 4). An instance-mask guided fusion module blends backbone features in non-instance regions with adapter features in instance regions (Eq. 5). Experiments on Cityscapes and a pedestrian-focused subset report improved FID and mIoU over OASIS, SDM, ControlNet, Uni-ControlNet, and FreestyleNet, along with qualitative evidence of clearer instance separation.","tokens_in":9381,"tokens_out":4313,"duration_ms":53474,"significance":"If the central claim is established, InstancePin addresses a real limitation of category-level layout-to-image diffusion: the lack of instance addressability in dense same-category regions. The design is practical: it avoids fine-tuning the pretrained backbone, uses an adapter for instance-specific control, and employs a mask-guided fusion to preserve global structure. The paper is clearly written and the proposed components are motivated. However, the experimental evidence currently does not separate the effect of the proposed coordinate-pinning mechanism from the effect of simply adding instance-map conditioning, and the reported metrics are category-level rather than instance-level. The method is a plausible contribution, but the quantitative support for the central 'instance disentanglement' claim needs substantial strengthening.","major_comments":[{"comment":"The comparison against baselines is confounded by an input asymmetry. InstancePin conditions on the instance map M and its derivatives P and B, while none of the baselines (OASIS, SDM, ControlNet, Uni-ControlNet, FreestyleNet) receive instance-level annotations. The reported gains (FID 40.3→37.6, mIoU 73.3→75.1 on Cityscapes; FID 46.6→43.5, mIoU 74.1→75.8 on Cityscapes-pedestrian) could therefore stem from the extra instance information rather than from coordinate pinning itself. The ablations in Table 2 all hold M fixed, so they cannot separate the value of the additional input from the value of the proposed architecture. Please add an instance-conditioned baseline, such as FreestyleNet or ControlNet fed the same instance mask/centers (e.g., via simple concatenation or an adapter without coordinate pinning), and/or an ablation that removes CPA while retaining P,B conditioning. Without t","section":"§4.2, Table 1"},{"comment":"No error bars, repeated seeds, or significance tests are reported. The FID and mIoU differences over FreestyleNet are modest (0.9–2.7 FID points, 1.0–1.8 mIoU points), and the ablation differences are similarly small. On a single evaluation it is impossible to tell whether these gains are stable or within run-to-run noise. Please report means and standard deviations over at least three seeds (or equivalent bootstrapped confidence intervals) for all numbers in Tables 1 and 2, and ideally state whether the differences are statistically significant. This is essential given that the paper's quantitative case rests on small margins.","section":"§4.1–§4.2, Tables 1–2"},{"comment":"The paper's stated goal is to reduce instance entanglement, but no instance-level metric is reported. mIoU measures category-level semantic consistency, and FID measures global fidelity; neither directly quantifies whether adjacent same-category instances are generated as distinct objects. The qualitative zoomed regions in Fig. 3 are suggestive but anecdotal. Please add an instance-level evaluation, for example: predicted instance segmentation mAP (using a pretrained Mask R-CNN or similar), Panoptic Quality, or a metric that measures whether the number of generated instances matches the ground-truth instance count in the layout. Such a metric is necessary to support the central 'instance-addressable' claim quantitatively.","section":"§4.2, §4.4"}],"minor_comments":[{"comment":"The attention maps in Fig. 4 are directly supervised by instance-region targets via Eq. (4), so their alignment with instance regions is a trained behavior, not an emergent property. The text in §4.3 should be tempered: it is a verification that the supervisory signal is learned, rather than evidence that coordinate pinning attention is inherently interpretable. This does not invalidate the method, but the framing should be adjusted.","section":"§4.3, Fig. 4"},{"comment":"The 'Cityscapes-pedestrian' subset is not precisely defined. Is it the subset of validation images containing pedestrians, or a cropping of pedestrian regions? Its size and selection criteria should be stated, as the reported numbers depend on this definition.","section":"§4.1"},{"comment":"The projection matrices W_Q, W_K, W_V are written without the scale index l, which is confusing since the feature F^l is scale-dependent. Clarify that these are per-scale projections, or explicitly denote them as W^l_Q etc.","section":"§3.3, Eq. (3)"},{"comment":"Several hyperparameters are not ablated or justified: the coordinate loss weight λ_coord=10, the number of Fourier frequency bands F, and the coordinate token dimension d_c. At least λ_coord should be examined, given that it controls the strength of the instance-region supervision that drives the central mechanism.","section":"§4.1"},{"comment":"No code or trained models are released. Given the reliance on a specific pretrained backbone (FreestyleNet) and detailed adapter architecture, releasing code would substantially aid reproducibility.","section":"§4.1"},{"comment":"Several references have formatting errors: Ref. [4] is missing author names after 'M., Heusel'; Ref. [17] contains a typo 'ZhoubTinghui, A'; Ref. [7] lists 'Ian, G.' rather than the full author name; Ref. [20] has inconsistent publisher capitalization ('Pmlr'). These should be corrected.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central methodological idea is sound and the paper is within the journal's scope. The main concern is the experimental confound: the comparison with baselines does not isolate the effect of coordinate pinning from the additional instance-map conditioning. This is fixable by adding instance-conditioned baselines and an ablation that removes CPA while keeping the extra input. The absence of any instance-level metric is also a significant gap for a paper whose title claims instance addressability. If the authors can address these points, the paper could become a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: the method is coherent and the problem is real, but the headline experiment is confounded. InstancePin conditions on instance maps M, deriving centers P and binary mask B, and uses all three at train and inference. The baselines (OASIS, SDM, ControlNet, Uni-ControlNet, FreestyleNet) only see semantic layout S and text. So the 2.7-point FID and 1.8-point mIoU gains over FreestyleNet could come from the extra instance-level conditioning itself, not from coordinate-pinning attention. The ablations hold M fixed, so they can't separate the value of the new input from the value of the mechanism. This is the load-bearing soft spot, and it's a real one.\n\nWhat's genuinely good: the design is well-thought-out. The instance-aware adapter keeps the pretrained backbone intact, the Fourier-encoded coordinate tokens are a sensible way to make instance centers queryable, and the mask-guided fusion confines adapter features to instance regions. The ablation logic is sound: removing the coordinate alignment loss (L_coord) causes a large drop, and removing IMGF hurts too. The qualitative figures do show cleaner separation of adjacent pedestrians. So there's a solid core here.\n\nThe other issues are secondary but worth naming. There are no error bars or repeated seeds; single-seed FID/mIoU differences of this size are within run-to-run noise in my experience. There's no instance-level metric, so mIoU doesn't directly measure the claimed disentanglement. The construction of the pedestrian-focused subset is unspecified. And the attention visualizations in Fig. 4 are partly circular: the attention maps are explicitly supervised with instance regions via Eq. 4, so seeing them align is expected, not emergent. None of these are fatal on their own; together they mean the evidence is thin.\n\nWho should read this: anyone working on layout-to-image diffusion or instance-aware controllable generation. The idea is worth engaging with, and the paper deserves a serious referee — but I'd condition acceptance on adding an instance-conditioned baseline (e.g., FreestyleNet or ControlNet fed the same B/P), plus variance estimates and an instance-level metric. Without those, the central claim stands only as a plausible hypothesis, not a demonstrated result.","headline":"Sensible architecture and a real problem, but the main comparison is confounded by an input asymmetry: baselines lack the instance maps that InstancePin consumes, so the gains in Table 1 don't isolate coordinate pinning.","tokens_in":9800,"tokens_out":2384,"would_cite":true,"duration_ms":32688,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"InstancePin pins each object to a coordinate anchor so layout-to-image diffusion can tell apart crowded same-category objects.","keywords":["layout-to-image generation","instance-aware diffusion","coordinate pinning attention","adapter-based control","Cityscapes","semantic image synthesis","instance separation"],"falsifier":"Run InstancePin and the FreestyleNet baseline over multiple training seeds and compute FID/mIoU confidence intervals, or evaluate with an instance-level metric such as per-instance segmentation of generated images matched to ground-truth instances. If the gains vanish within noise, or if per-instance matching accuracy is no better than the category-level baseline, the claim that coordinate pinning achieves instance disentanglement is not established.","tokens_in":9037,"feed_emoji":"🖼️","tokens_out":2544,"duration_ms":34616,"temperature":0.7,"pith_summary":"Category-level layout control tells a diffusion model where a semantic class appears, but not how neighboring instances of that class should be kept separate. InstancePin argues that encoding each instance's center coordinate as a queryable token, and supervising the resulting attention maps with instance regions, makes generation instance-addressable while leaving the pretrained category-level backbone intact. On Cityscapes, the method improves FID from 40.3 to 37.6 and mIoU from 73.3 to 75.1 over the strongest baseline, with larger gains on a pedestrian-focused subset. If the claim holds, dense urban scenes with many small, similar objects become controllable at the level of individual objects instead of shared semantic blobs.","feed_headline":"Coordinate anchors keep crowded pedestrians distinct in diffusion images","feed_subtitle":"On Cityscapes, FID drops from 40.3 to 37.6 and mIoU rises to 75.1 over the strongest baseline.","key_machinery":"Coordinate Pinning Attention is the central mechanism: each instance center is converted into a coordinate token via Fourier features and a small mapping network, and latent image features cross-attend to these tokens so that every spatial location can be associated with a specific instance. The coordinate alignment loss forces each token's attention map to activate its own instance region, and the Instance-Mask Guided Fusion module combines the resulting adapter features with frozen backbone features using a binary instance mask. Together these components separate same-category instances while preserving the pretrained model's global semantic prior.","core_discovery":"The central claim is that coordinate-pinned instance anchors, combined with mask-guided feature fusion, let a layout-to-image diffusion model treat each object instance as a separately addressable unit without degrading category-level semantic control. Rather than injecting instance masks into the pretrained denoising backbone, InstancePin adds an independent instance-aware adapter: instance centers are encoded with Fourier features and projected into coordinate tokens, latent image features query these tokens through coordinate pinning attention, and the resulting attention maps are explicitly supervised to align with instance regions. An instance-mask guided fusion module then routes backb","pith_inferences":["The coordinate-token design suggests a direct route to instance-level editing: by manipulating a single token's value or attention map, an editor could localize changes to one object without touching its neighbors or the background.","Because the method only needs center coordinates and a binary instance mask, it should transfer to other instance-annotated layout domains such as indoor scenes or autonomous-vehicle datasets, though the paper only tests Cityscapes.","The absence of an instance-level evaluation metric leaves open whether the FID/mIoU gains reflect true per-instance separation or simply better global fidelity; a per-instance matching metric would settle that question.","For heavily occluded or overlapping instances, a single center coordinate may not uniquely identify an object, so the anchor representation may need to be extended to multiple keypoints per instance in such settings."],"forward_implications":["Adjacent same-category objects in dense urban layouts can be generated with distinct boundaries and appearances instead of merged, averaged regions.","Instance-level control can be added to a frozen pretrained diffusion backbone through an independent adapter, avoiding the need to retrain the category-level model.","Coordinate attention maps become spatially interpretable, indicating which latent region corresponds to which instance anchor during denoising.","The pedestrian-focused subset shows that gains concentrate exactly where instance entanglement is most severe, supporting the method's stated purpose."],"supporting_citations":[{"why":"Supplies the pretrained FreestyleNet backbone that InstancePin builds on and the strongest baseline it must beat.","marker":"[29]"},{"why":"Provides the copied-branch control architecture that the Instance-Aware Adapter generalizes for instance-level control.","marker":"[31]"},{"why":"Represents the unified multi-condition control approach that InstancePin distinguishes itself from and compares against.","marker":"[34]"},{"why":"Introduces learnable attention layers for grounded text-to-image generation, a direct precedent for querying spatial anchors.","marker":"[11]"},{"why":"Establishes the latent diffusion framework underlying the pretrained generation pipeline.","marker":"[21]"},{"why":"Defines the FID metric used to measure image fidelity in the central comparisons.","marker":"[4]"},{"why":"Provides the DRN-D-105 segmentation model used to compute the mIoU semantic consistency metric.","marker":"[30]"}],"fun_headline_variants":["Coordinate pinning keeps each pedestrian distinct in diffusion images","InstancePin: coordinate pins for instance-addressable layout-to-image","Pinning coordinates separates crowded instances in diffusion images","Fourier-pinned anchors resolve instance overlap in layout-to-image"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The central claim rests on the assumption that FID and mIoU differences of roughly one to three points, measured on a single evaluation without error bars or an instance-level metric, reflect genuine per-instance separation rather than run-to-run noise or improved global fidelity.","fun_headline_variants_meta":{"raw":{"variants":["Coordinate pinning keeps each pedestrian distinct in diffusion images","InstancePin: coordinate pins for instance-addressable layout-to-image","Pinning coordinates separates crowded instances in diffusion images","Fourier-pinned anchors resolve instance overlap in layout-to-image"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000618,"raw_usage":{"total_tokens":2717,"prompt_tokens":767,"completion_tokens":1950,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":1882}},"tokens_in":511,"tokens_out":1950,"duration_ms":17819,"temperature":1.0,"reasoning_tokens":1882,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T00:30:42.208330+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run InstancePin and the FreestyleNet baseline over multiple training seeds and compute FID/mIoU confidence intervals, or evaluate with an instance-level metric such as per-instance segmentation of generated images matched to ground-truth instances. If the gains vanish within noise, or if per-instance matching accuracy is no better than the category-level baseline, the claim that coordinate pinning achieves instance disentanglement is not established.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained FreestyleNet backbone that InstancePin builds on and the strongest baseline it must beat."},{"cited_title":"Advances in Neural Information Processing Systems36, 11127–11150 (2023)","cited_arxiv_id":null,"evidence_quote":"Represents the unified multi-condition control approach that InstancePin distinguishes itself from and compares against."},{"cited_title":"In: Proc","cited_arxiv_id":null,"evidence_quote":"Defines the FID metric used to measure image fidelity in the central comparisons."}],"review_version":1}