{"id":"b5c0f272-2488-410d-8db7-e1a9b2a67060","arxiv_id":"2411.16783","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"CoCoNO improves prompt-image alignment by optimizing the initial latent with attention complete and contrast losses, reducing subject neglect and mixed-subject artifacts.","lead":"CoCoNO optimizes the starting noise of a text-to-image diffusion model to avoid two common failures: a prompt subject disappearing, or two subjects blending into one mixed creature. It works with a pretrained model and needs no extra training, so it can be dropped into existing generation pipelines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"As written, Eq. (3) makes the optimized 'initial latent' a rescaled one-step denoised image rather than a Gaussian noise sample, which breaks the initial-noise-optimization premise and miscalibrates the KL loss.","rationale":"The reader's weakest assumption is that one-step attention maps reliably predict final subject placement. That is a real concern, but the more basic issue is that, as written, the optimization loop does not appear to maintain a valid initial noise distribution at all. Equation (3) constructs the next 'noise' input from z_{T-1}, a partially denoised latent, so the object being optimized is not a fresh Gaussian sample. This directly undermines the claim that CoCoNO is an initial noise optimization method and also means the KL regularization in Eq. (7) is miscalibrated. If Eq. (3) is a typo and the authors intended a standard reparameterization with fixed ε, then the reader's one-step proxy concern becomes the main issue; but in the manuscript as provided, the update rule is the most load-bearing weakness. I therefore keep the CONDITIONAL verdict, but I would condition acceptance on correcting or defending Eq. (3) and showing that the gains persist under a valid reparameterization. The proposed test would separate the contribution of the attention losses from the artifact of the stated update. This is not an accusation of bad faith; it is an internal consistency check that the authors can resolve with a small code change and rerun.","tokens_in":14387,"tokens_out":7628,"duration_ms":83414,"concrete_test":"Rerun CoCoNO with the correction z'_T = μ' + σ' ε for a fixed ε ~ N(0, I), keeping the losses, optimizer, evaluation protocol, and baselines identical. If the reported gains in Figure 7 and Table 1 largely disappear or shrink substantially, the headline result depends on the stated Eq. (3) update rather than on the attention losses themselves. As a second check, compute the empirical mean and variance of the final optimized z_T over the 64 seeds per prompt; if these deviate markedly from 0 and 1, the KL constraint is not satisfied by the actual starting latent distribution.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that CoCoNO improves initial noise optimization. Section 3.3 defines the update as z'_T = μ' + σ' z_{T-1}, where z_{T-1} is obtained by one-step denoising. Even if μ and σ are updated using the proposed losses, the next input to the denoiser is not reparameterized fresh noise; it is a scaled version of a partially denoised image. The KL loss in Eq. (7) is written as KL(N(μ, σ^2) || N(0,1)), which assumes the actual z_T is Gaussian noise with variance σ^2. But the variance of σ' z_{T-1} is σ'^2 Var(z_{T-1}), which is not estimated and is generally not 1. Thus the KL term does not enforce the stated constraint, and the reported improvements may come from feeding an out-of-distribution, structured latent into the denoiser (latent refinement) rather than from the attention contrast and complete losses optimizing a valid initial noise. This also makes the comparison with InitNO and other noise-optimization baselines inequitable if those baselines use a standard reparameterization.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoCoNO, a training-free method for optimizing the initial latent of Stable Diffusion v2.1 for text-to-image generation. Two attention-based losses are introduced: an attention complete loss, which encourages each subject token to own a distinct high-response segment in the self-attention map, and an attention contrast loss, which penalizes overlap between one subject's cross-attention map and another subject's self-attention segment. These losses are combined with a KL regularization term and minimized by updating the parameters of the latent distribution over repeated one-step denoising iterations. The authors evaluate on T2I-CompBench, GenAI-Bench, and a newly curated 20-prompt Complex-Bench, reporting higher VQAScore and text-text CLIP similarity than Stable Diffusion, InitNO, CONFORM, Divide-and-Bind, and Attend-and-Excite, along with a user study favoring CoCoNO.","tokens_in":14593,"tokens_out":7445,"duration_ms":67980,"significance":"If the empirical claims hold, CoCoNO would be a useful training-free contribution to improving subject coverage and reducing subject mixing in text-to-image generation. The problem identification is clear, and the two losses address a genuine limitation of prior attention-based methods. The paper has notable strengths: the central evaluation metrics (VQAScore and text-text CLIP similarity) are independent of the proposed losses, and the ablation in Table 1 shows that each loss contributes to the reported gain. However, the load-bearing update rule in Eq. (3) does not produce a valid Gaussian initial noise sample, and the headline comparisons lack error bars and significance tests. These issues are substantial but appear addressable, so the contribution remains potentially significant after revision.","major_comments":[{"comment":"The updated latent z'_T = mu' + sigma' z_{T-1} is not a sample of the Gaussian initial noise distribution that the method claims to optimize. Since z_{T-1} is the output of one-step denoising, sigma' z_{T-1} is a scaled, structured latent rather than an independent Gaussian sample; its variance is sigma'^2 Var(z_{T-1}), which is not controlled by the KL loss in Eq. (7). The KL term KL(N(mu, sigma^2) || N(0,1)) regularizes the parameters, not the actual distribution of z'_T, so the stated constraint is not enforced. This changes the interpretation from initial noise optimization to a form of iterative latent refinement and could make comparisons with noise-optimization baselines inequitable. Please either reparameterize with fresh noise epsilon ~ N(0,I) at each iteration, or revise the framing and the KL loss to match the actual distribution of the updated latent.","section":"Section 3.3, Eq. (3)"},{"comment":"The core premise that attention maps computed from a single one-step denoised latent are causally predictive of final image composition is supported only by qualitative examples. If one-step attention maps do not reliably indicate where subjects will appear after full denoising, Eqs. (5)-(6) optimize an auxiliary quantity whose transfer to the final image is not guaranteed. The authors should quantify this relationship, for example by comparing segments obtained from one-step attention maps with final-image subject masks from Grounded-SAM across a sample of prompts and reporting agreement as a function of timestep or optimization iteration. This would also justify the choice of one-step denoising rather than a later timestep.","section":"Section 3.2 and Figure 4"},{"comment":"The headline quantitative results are averages over 64 generated images per prompt, with only 20 prompts for Complex-Bench, and no variance, confidence intervals, or significance tests are reported. Without these, the claim of significant improvement over baselines is not established; the differences in Figure 7 may be within seed-level noise. Please report per-prompt distributions, standard errors, and pairwise significance tests (for example bootstrap or paired tests), and specify the number of prompts per benchmark for the aggregated numbers in Table 1.","section":"Figure 7 and Table 1"},{"comment":"The losses as written divide by the sum of A_s^i over its spatial dimensions. When a subject has a missing self-attention segment, the paper explicitly creates zero-element matrices for that segment, making the denominator zero and the loss expression undefined (0/0 for a missing diagonal segment) unless a numerical epsilon is added. Please define the normalization for missing segments, and clarify whether the assignment permutation is recomputed after zero segments are populated during optimization.","section":"Equations (5)-(6)"},{"comment":"The method depends on several empirical choices without sensitivity analysis: lambda_1=1, lambda_2=1, lambda_3=500, alpha=16, beta=0.5, Adam learning rate 10^{-2}, restart thresholds, and the latent cache. Given that lambda_3 is three orders of magnitude larger than the attention losses and that the KL loss is miscalibrated under Eq. (3), it is important to show that the reported performance is not a knife-edge artifact of a particular weight combination. A sensitivity sweep over at least the loss weights and alpha/beta would substantially strengthen the claim.","section":"Section 3.3 and Appendix A.2"}],"minor_comments":[{"comment":"The abstract contains a duplicated word ('despite despite') and the title in the full text has a spacing error ('No ise'); these should be corrected.","section":"Abstract"},{"comment":"The notation 'P ⊗ C' is described as matrix multiplication, but the tensor-product symbol is commonly read as elementwise or Kronecker product; please use a standard matrix product notation such as P C.","section":"Section 3.3"},{"comment":"The figure contains garbled text ('K  VQK  VQ') and the formulas for the loss computation and latent update are hard to read; please redraw it with clear notation.","section":"Figure 5"},{"comment":"Table 1 does not state whether the reported scores are averaged over all three benchmarks or a subset; please state this explicitly and give the number of prompts used for each aggregated value.","section":"Section 4.2"},{"comment":"The user study reports only the percentage of preferences; please also report the number of participants, the number of comparisons per participant, and inter-rater agreement.","section":"Section 4.2, Table 2"},{"comment":"The Grounded-SAM mask-count and overlap evaluation measures almost exactly what the proposed losses are designed to enforce, so it is not an independent evaluation metric; please present it as a diagnostic of the intended mechanism rather than as evidence that general text-image alignment improves.","section":"Appendix A.3, Figure 10"},{"comment":"Reference [2] lists only 'Tom B Brown' as the author; this should be 'Brown et al.' with the full author list.","section":"References"},{"comment":"The sentence 'from The subject tokens use during the latent optimization...' contains a grammatical error and should be rewritten.","section":"Appendix A.2"}],"recommendation":"major_revision","confidential_remarks":"The Eq. (3) issue is the main technical concern: as written, the method refines a partially denoised latent rather than optimizing a valid initial Gaussian noise sample, and the KL loss is miscalibrated. This is fixable by reparameterizing with fresh noise or by reframing the method as latent refinement and adjusting the KL term accordingly. The statistical presentation also needs strengthening before the headline claims can be accepted. The paper does not provide code or a reproducibility artifact, which limits verification of the reported numbers. If the authors address the update rule and add significance/robustness analysis, the contribution could be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CoCoNO is a genuine extension of InitNO, and the two losses are the right thing to try if you believe one-step attention maps are predictive of final composition. The Hungarian mapping between self-attention segments and subject cross-attention maps, with zero-padding for missing segments, is a clean idea, and the ablation (Table 1) shows each loss helps. The main evaluation uses independent metrics (VQAScore, text-text CLIP), and the user study strongly favors CoCoNO. That is real evidence, and the authors are honest about the limits (no inter-subject relations, Appendix A.5).\n\nThe soft spots are in the formulation and the evidence quality. The stress-test note is correct: Eq. (3) sets z'_T = mu' + sigma' z_{T-1}, where z_{T-1} is a one-step denoised latent. That is not a reparameterized Gaussian sample, and its variance is not sigma'^2. The KL loss in Eq. (7) is computed on the parameters (mu, sigma), not on the actual latent, so it does not enforce what it claims. The method is therefore better described as latent refinement than initial noise optimization. If InitNO uses the same update rule—the paper says it does—the relative comparison may still be fair, but the central premise of the paper is misstated. This is a load-bearing framing issue, not a typo.\n\nThe empirical claims also need tightening. Figure 7 reports 64-image averages with no error bars or significance tests; the loss weights are fixed (lambda1=lambda2=1, lambda3=500) with no sensitivity analysis; and the restart-and-cache procedure selects a best-of-many run, which can inflate the gap. The Grounded-SAM evaluation in Figure 10 measures mask counts and overlap, which are close to what the losses directly optimize, so it is partially circular as evidence for the final-image claim. No code is released, and Complex-Bench is only 20 prompts.\n\nWho is this for? Researchers working on test-time guidance and compositional generation will find the segment-to-subject assignment idea worth engaging with. But the paper needs major revision before the numbers can be taken at face value: fix or reframe Eq. (3), add error bars and a sensitivity analysis, and release code. I would not desk-reject it—the idea is solid enough for a serious referee—but I would expect heavy revision.","headline":"A promising attention-based fix for subject mixing and neglect, but Eq. (3) quietly turns 'initial noise optimization' into latent refinement, so the paper needs a major reframing before the numbers are trustworthy.","tokens_in":15190,"tokens_out":5503,"would_cite":false,"duration_ms":48278,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoCoNO claims that two attention-map defects explain most subject neglect and mixing in text-to-image models, and that optimizing the initial noise to fix them yields better aligned images than existing methods.","keywords":["initial noise optimization","text-to-image synthesis","attention maps","subject neglect","subject mixing","diffusion models","latent optimization","attention contrast loss"],"falsifier":"A concrete test: for prompts with two subjects, compute the one-step attention maps and check whether the CoCoNO segment conditions hold, then fully denoise and measure subject presence and mixing with a segmentation model such as Grounded-SAM. If a large fraction of latents whose one-step maps satisfy both conditions still produce missing or mixed subjects, the proxy assumption is false. A complementary check would perturb only the contrast term (or only the complete term) and observe whether the predicted mixing (or neglect) appears.","tokens_in":14156,"feed_emoji":"🖼️","tokens_out":5834,"duration_ms":47360,"temperature":0.7,"pith_summary":"CoCoNO claims that the remaining failures of text-to-image generation — subjects left out entirely (subject neglect) and subjects blended into one another (subject mixing) — have two specific attention-map causes: missing self-attention segments for some prompt tokens (attention neglect) and one token's cross-attention leaking into another token's self-attention segment (attention interference). The paper's contribution is a noise-optimization algorithm that fixes both causes at once by jointly reading self- and cross-attention maps on one-step denoised latents. Its two losses, attention complete and attention contrast, force each subject to own a distinct high-response segment and forbid cross-attention from spilling into other subjects' segments. On three benchmarks (T2I-CompBench, GenAI-Bench, and a custom Complex-Bench) and in a user study, CoCoNO reports larger image-text and text-text similarity scores than InitNO, CONFORM, Divide-and-Bind, Attend-and-Excite, and plain Stable Diffusion, with no model retraining.","feed_headline":"Two attention losses fix missing and mixed subjects in text-to-image","feed_subtitle":"CoCoNO optimizes the starting noise so every prompt subject gets its own attention segment.","key_machinery":"The central machinery is a pair of losses computed from aggregated self- and cross-attention maps of a one-step denoised latent. After sigmoid-softening and segmenting the self-attention map (Otsu thresholding plus connected components), a Hungarian assignment maps each of the n subject tokens to one of the n self-attention segments, padding with zero segments when the map has fewer than n segments. The attention complete loss maximizes the smallest diagonal intersection (Eq. 6), forcing every subject to have a present, high-response segment, while the attention contrast loss minimizes off-diagonal intersections (Eq. 5), preventing cross-attention interference. These losses, plus a KL term to stay near N(0,1), update the mean and covariance of the initial latent; the updated latent is denoised one step, and the loop repeats.","core_discovery":"The central claim, stated by the authors, is that attention neglect and attention interference — not merely high-response but non-overlapping attention maps — are the decisive causes of subject omission and subject mixing after initial-noise optimization, and that correcting them during initialization propagates to the final image. They demonstrate that when each subject token is assigned a dedicated self-attention segment (complete) and no token's cross-attention activates another subject's segment (contrast), the optimized latent denoises into images containing all prompt subjects with their own properties. This is established through quantitative comparisons on three benchmarks and a four-way user study in which CoCoNO is preferred 72.6% of the time.","pith_inferences":["The same contrast-complete principle could be applied to the cross-attention maps during the denoising trajectory itself, not only at initialization, which might correct misplacements that appear only in later sampling steps.","The hard Hungarian assignment assumes a fixed number of subject tokens; extending to variable-length prompts or modeling inter-subject relations (a limitation the authors name) may require a soft assignment or an auxiliary relation model.","The loss values themselves could serve as a diagnostic: a prompt that keeps the complete loss near one after optimization is likely structurally unrenderable by the backbone, giving users early feedback on prompt feasibility."],"forward_implications":["CoCoNO requires no retraining: it optimizes only the initial noise's mean and covariance, so it is directly compatible with any latent diffusion backbone.","The attention complete loss creates self-attention segments that were entirely absent, directly targeting the case where a subject is omitted; the attention contrast loss prevents cross-attention leakage, targeting mixed properties.","Because each subject token is assigned to exactly one segment by the Hungarian step, the method naturally enforces a one-to-one subject-to-region mapping.","Reported inference time rises from 7.71s (Stable Diffusion) to 17.21s (CoCoNO) per 512x512 image, a modest add-on for test-time alignment gains."],"supporting_citations":[{"why":"The initial-noise-optimization baseline that CoCoNO builds on and compares against; supplies the latent update framework.","marker":"[6]"},{"why":"Prior cross-attention segregation work that motivates the use of cross-attention maps for subject retention.","marker":"[1]"},{"why":"Provides the cross-attention aggregation and gaussian smoothing used in the pipeline, and is a test-time baseline.","marker":"[3]"},{"why":"The Hungarian assignment optimization that maps each self-attention segment to a subject token.","marker":"[16]"},{"why":"One of the evaluation benchmarks used to measure alignment.","marker":"[11]"},{"why":"The image-text evaluation benchmark and metric (VQAScore) used in comparisons.","marker":"[20]"},{"why":"A comparative baseline in experiments.","marker":"[23]"},{"why":"A comparative baseline in experiments.","marker":"[19]"}],"fun_headline_variants":["CoCoNO: Two attention losses fix missing and mixed subjects","Optimize initial noise to stop subject neglect and interference","Attention contrast and complete ensure all prompt subjects appear","CoCoNO: Attention losses end subject omission and mixing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that attention maps taken from a single one-step denoised latent reliably predict where subjects will end up in the fully denoised image; the paper motivates this with examples but does not quantify the correlation.","fun_headline_variants_meta":{"raw":{"variants":["CoCoNO: Two attention losses fix missing and mixed subjects","Optimize initial noise to stop subject neglect and interference","Attention contrast and complete ensure all prompt subjects appear","CoCoNO: Attention losses end subject omission and mixing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000251,"raw_usage":{"total_tokens":1541,"prompt_tokens":910,"completion_tokens":631,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":566}},"tokens_in":526,"tokens_out":631,"duration_ms":6113,"temperature":1.0,"reasoning_tokens":566,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:27:12.719130+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: for prompts with two subjects, compute the one-step attention maps and check whether the CoCoNO segment conditions hold, then fully denoise and measure subject presence and mixing with a segmentation model such as Grounded-SAM. If a large fraction of latents whose one-step maps satisfy both conditions still produce missing or mixed subjects, the proxy assumption is false. A complementary check would perturb only the contrast term (or only the complete term) and observe whether the predicted mixing (or neglect) appears.","supporting_citations":[{"cited_title":"Initno: Boosting text-to-image diffu- sion models via initial noise optimization","cited_arxiv_id":null,"evidence_quote":"The initial-noise-optimization baseline that CoCoNO builds on and compares against; supplies the latent update framework."},{"cited_title":"A-star: Test-time attention segregation and retention for text-to-image synthesis","cited_arxiv_id":null,"evidence_quote":"Prior cross-attention segregation work that motivates the use of cross-attention maps for subject retention."},{"cited_title":"Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the cross-attention aggregation and gaussian smoothing used in the pipeline, and is a test-time baseline."},{"cited_title":"T2i-compbench: A comprehensive bench- mark for open-world compositional text-to-image genera- tion","cited_arxiv_id":null,"evidence_quote":"One of the evaluation benchmarks used to measure alignment."},{"cited_title":"Conform: Contrast is all you need for high- fidelity text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"A comparative baseline in experiments."},{"cited_title":"Divide & bind your attention for improved generative seman- tic nursing","cited_arxiv_id":null,"evidence_quote":"A comparative baseline in experiments."}],"review_version":1}