{"id":"d794b969-ad2a-4a2e-9111-9ffa6ca55e4d","arxiv_id":"2508.21090","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Q-Align aligns queries between two images and rearranges keys and values to suppress attention leakage in zero-shot appearance transfer.","lead":"A new attention-control technique, Q-Align, reduces attention leakage in zero-shot appearance transfer by matching queries between the structure and appearance images and using that match to rearrange keys and values. The method reports higher appearance fidelity and structure preservation than the Cross-Image baseline and other diffusion-based transfer methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core premise that query-query alignment gives correct spatial correspondence is supported only by one qualitative example; with k=1 hard selection, any error in S directly corrupts K/V and the paper reports no quantitative alignment accuracy.","rationale":"The reader identified Q-Q alignment correctness as the weakest assumption; I agree and sharpen it. The paper's own evidence for this premise is one qualitative attention-grid example (Fig. 3) plus a citation about query/key embedding spaces, which is not a demonstration of cross-image query comparability. The mechanism amplifies any error because k=1 removes the averaging that might otherwise mask misassignments, and the diagonal fallback (Eq. 4) preserves the original Cross-Image behavior at uncovered positions. The evaluation does not close this gap: the GPT-4o score is self-proposed without human-correlation validation, the quantitative margins over Cross-Image are small (e.g., 4.30 vs 4.19 on the appearance score), no significance tests are given, and only 65/94 leakage cases are improved. These factors reinforce, rather than replace, the core concern about whether S is actually reliable. A correspondence accuracy check (PCK-style) directly tests the premise and would settle whether the mechanism works as claimed. If that check passes, the method is plausible; if it fails, the central claim is unsupported. Since the reader's verdict already conditions on exactly this uncertainty, I recommend no change to the CONDITIONAL verdict.","tokens_in":8788,"tokens_out":6844,"duration_ms":81746,"concrete_test":"On a random subset of the 180 pairs, annotate or obtain pseudo ground-truth correspondences (e.g., via DINO/RAFT or manual keypoints), then compute the top-1 accuracy of the mapping induced by S = Q_app Q_str^T at the layers and timesteps used by Q-Align, and compare with the Q-K mapping of Cross-Image. If Q-Q top-1 accuracy is not substantially higher, or if the fraction of P' rows falling back to the diagonal (Eq. 4) is large and does not correlate with worse outputs, the proposed mechanism is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism rests on S = Q_app Q_str^T (Eq. 2) being a semantically correct dense correspondence. Eq. 3-7 build a hard k=1 rearrangement matrix P' from S and use it to replace K_app,V_app before Eq. 8. If S misassigns a query, the wrong appearance key/value is placed at that location with no averaging to dilute the error. Moreover, any str position not chosen as top-1 by any app query falls back to the diagonal (Eq. 4), i.e., the original Cross-Image key/value at that same index—exactly the leakage-prone configuration the method claims to remove. The only evidence for S's correctness is the qualitative 3x3-grid example in Fig. 3 and a citation [39] that queries and keys occupy different embedding spaces. That citation does not establish that queries from two different images are directly comparable, and no quantitative correspondence metric (e.g., PCK) or ablation on k is reported. The leakage statistics are also partial: leakage occurs in 94/180 pairs and Q-Align improves only 65 of those, leaving failures unexplained. Thus the central improvement mechanism is currently unvalidated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies 'attention leakage' in zero-shot appearance transfer using Cross-Image attention and proposes Q-Align to mitigate it. The method replaces query-key alignment with query-query alignment S = Q_app Q_str^T, constructs a top-1 rearrangement matrix P' from S, rearranges K_app and V_app into K* and V*, and applies the cross-image attention softmax(Q_out K*^T / sqrt(d)) V*. The authors claim improved appearance fidelity and competitive structure preservation over Cross-Image, DragonDiffusion, DiffuseIT, and SpliceViT, and introduce a GPT-4o-based evaluation protocol for appearance fidelity and structural consistency.","tokens_in":9064,"tokens_out":3351,"duration_ms":41912,"significance":"The problem is relevant: attention leakage is a real artifact in training-free appearance transfer, and the proposed mechanism is simple, zero-shot, and accompanied by public code. The paper also contributes a useful analysis of query-key versus query-query alignment in Stable Diffusion. However, the central claim rests on two currently weak pillars: (i) the assumption that S provides semantically correct dense correspondence is backed only by one qualitative example and not by any quantitative correspondence metric, and (ii) the main quantitative evidence is a new self-proposed GPT-4o metric that is not validated against human raters, while pre-existing Gram-loss and IoU metrics show mixed or inconclusive results. If these gaps are filled, the method would be a solid contribution; in its current form the evidence is not yet commensurate with the claims.","major_comments":[{"comment":"The mechanism hinges on S = Q_app Q_str^T being a semantically correct dense correspondence. With k=1, each output location receives keys/values from exactly one appearance location, so any mis-assignment in S directly places incorrect appearance features with no averaging to dilute the error. The paper provides only the single 3×3 grid of Fig. 3 as evidence, and the cited [39] supports a different claim about query/key spaces, not that queries from two different images are directly comparable. Please report a quantitative correspondence accuracy (e.g., PCK on annotated pairs) and an ablation over k. Also, the fallback in Eq. (4), if triggered, reverts to the original diagonal K/V, which is exactly the leakage-prone configuration; the frequency of that fallback is not reported.","section":"§4.2, Eqs. (3)–(8)"},{"comment":"The main quantitative advantage of Q-Align is measured by a new GPT-4o-based score that the paper itself proposes. The only support that this score matches human perception is Figure 5, which is anecdotal and reports no correlation, agreement, or inter-rater statistic. Since both the method and the metric are new, the reported gains (e.g., average 4.30 vs. 4.19 for Cross-Image) could reflect biases in the scoring procedure. Please validate the protocol against a human-rated sample quantitatively, or report results on established metrics that are known to correlate with perception.","section":"§5.1, Tables 2–3"},{"comment":"On the pre-existing metrics, Q-Align is not consistently better: Gram loss is worse than Cross-Image in Animal (1.68 vs. 1.08) and Car (1.32 vs. 1.12), and the IoU improvements are small relative to the reported standard deviations (e.g., Animal 0.71/0.12 vs. 0.66/0.16). The paper dismisses Gram loss as unreliable, but the supporting argument is one illustrative example rather than a systematic analysis. Please provide statistical significance tests and, ideally, additional established perceptual metrics (e.g., LPIPS) to substantiate the claimed appearance-fidelity advantage.","section":"§5.1, Table 1"},{"comment":"The leakage analysis reports that leakage occurs in 94/180 pairs and Q-Align improves 65 of those. This means 29 leakage cases are not improved, and there is no breakdown of how often Q-Align leaves leakage unchanged, worsens it, or introduces new artifacts in the 86 non-leakage or non-improved pairs. Since the method introduces a hard spatial selection, error analysis on the failure cases is needed to understand when the query-query alignment fails and whether the failures correlate with object size, background complexity, or image-pair similarity.","section":"§3, attention-leakage statistics"}],"minor_comments":[{"comment":"The notation is confusing: (P^T) is defined but then P is used in Eqs. (4)–(7). Please clarify the orientation of the matrix and why the transpose form is introduced.","section":"§4.2, Eq. (3)"},{"comment":"The figure illustrates that Gram loss disagrees with the GPT score, but only for one selected example. A quantitative comparison on a larger sample would strengthen the argument for the proposed metric.","section":"§5.1, Figure 5"},{"comment":"The qualitative selection explicitly chooses samples that exhibit leakage in Cross-Image. Please include a set of randomly selected pairs to control for selection bias.","section":"§5.2, Figure 6"},{"comment":"The threshold 'values below 0.2 are masked as 0' is arbitrary. Please state the choice criterion or provide a sensitivity analysis.","section":"§5.3, Figure 7"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely acceptable for a BMVC-style venue after the evaluation is strengthened. My main concern is that the central mechanism is unvalidated: the query-query correspondence assumption is supported only by a qualitative example, and the main quantitative improvement is on a self-proposed GPT-4o metric without human validation. I would advise the editor to require a correspondence-accuracy study or a k-ablation, and a human-rated evaluation of the GPT metric, before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: Q-Align is a sensible, zero-shot tweak to Cross-Image attention for appearance transfer. It rearranges the keys and values of the appearance image according to a query-query similarity map between structure and appearance images, before doing the usual cross-image attention. The idea is simple, the paper is clearly written, and the qualitative results do show sharper attention maps and fewer leakage artifacts. I believe them that this helps.\n\nWhat's actually new: the query-query alignment idea itself is not new (see [39] and others), but applying it to permute/aggregate K and V in this cross-image setting is a new combination. The implementation is straightforward and the code is promised.\n\nWhere it gets soft: the quantitative case is weaker than the qualitative one. The headline metric is a GPT-4o score they introduce in the same paper, with no validated agreement against human raters shown anywhere. The gains are small: 4.30 vs 4.19 on appearance, 4.31 vs 4.21 on structure, on a 1-5 scale. On the pre-existing Gram loss and IoU metrics, the comparison is mixed: average Gram loss is the same (2.08 vs 2.09), and in animal/animal-face/car domains Q-Align is worse. Only the IoU average is consistently better, but standard deviations are large.\n\nThe central mechanism also worries me. S = Q_app Q_str^T is assumed to give semantically correct spatial correspondence. They show one 3x3 grid example and cite [39] to argue queries from different images are comparable to each other, not to keys. That's plausible, but it's not tested quantitatively (no PCK or similar). And because they use hard top-1 selection, a wrong top-1 directly places a wrong key/value at that position. The fallback diagonal (Eq. 4) keeps the original key/value for positions that aren't matched, which is exactly the leakage-prone configuration from Cross-Image. They report 94/180 pairs leaked in Cross-Image, and Q-Align improves 65 of those; 29 failures are not analyzed.\n\nWould I accept it? Yes, but with major revision. The core idea is worth a serious referee. The authors need to validate the GPT metric against human raters, report statistical significance, ablate k, and show a quantitative correspondence accuracy for the query-query map. It's a fair BMVC-quality paper; I just wouldn't cite the evaluation as evidence without those additions.","headline":"A plausible, low-cost fix for Cross-Image attention leakage, but the headline gains rest on an unvalidated GPT metric and mixed old-metric results.","tokens_in":9517,"tokens_out":2477,"would_cite":true,"duration_ms":28286,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Query-query alignment, not query-key alignment, stops attention leakage in zero-shot appearance transfer.","keywords":["attention leakage","appearance transfer","query-query alignment","diffusion models","zero-shot image editing","attention control","key-value rearrangement","semantic correspondence"],"falsifier":"Take image pairs with ground-truth dense semantic correspondence and compare the accuracy of top-1 query-query matches against top-1 query-key matches; if query-query alignment is not systematically more accurate, the core premise fails. A complementary check: run Q-Align on pairs with extreme pose or category differences and look for outputs where the structure is preserved but the appearance identity lands on semantically wrong regions, which would indicate the alignment matrix is unreliable.","tokens_in":8704,"feed_emoji":"🎨","tokens_out":9921,"duration_ms":97938,"temperature":0.7,"pith_summary":"Appearance transfer re-paints one image's texture and identity onto another image's structure. The paper identifies a failure mode in the standard cross-image attention recipe: it matches the output image's queries against the appearance image's keys, and because queries and keys are not directly comparable, attention spreads into wrong regions—objects leak onto backgrounds and vice versa. The proposed method, Q-Align, instead aligns queries across the two images, uses the best match per query to rearrange the appearance image's keys and values, and runs attention with those rearranged key/value pairs. This requires no optimization, no extra training, and no masks. The authors report that Q-Align reduces leakage and improves appearance fidelity over previous zero-shot methods while keeping structure intact, and they support the comparison with a vision-language-model-based scoring protocol designed to track human judgment.","feed_headline":"Query-query alignment tames leaky zero-shot appearance transfer","feed_subtitle":"Rearranging keys and values by query matches keeps object textures on objects and backgrounds on backgrounds.","key_machinery":"The load-bearing mechanism is the query-query alignment matrix S = Q_app Q_str^T together with the aggregation matrix P' derived from its per-row top-k (k=1) selection. S states which appearance-image position corresponds to each structure-image position. P' encodes those matches as sparse weights, row-wise softmax reweights them, and applying P' to the appearance keys and values produces K* and V*. These rearranged key/value pairs are then consumed by the standard cross-image attention formula. The design choice of rearranging keys and values rather than queries is deliberate, since queries carry the spatial semantics of the output and should not be moved. The whole procedure runs in a sing","core_discovery":"Q-Align's central claim is that attention leakage in zero-shot appearance transfer comes from query-key misalignment: when the output image's queries are dotted against the appearance image's keys, attention spreads over semantically related but wrong regions. The paper replaces that signal with a query-query alignment matrix S = Q_app Q_str^T, keeps the top-1 match in each row, and turns those matches into a sparse aggregation matrix P' via row-wise softmax. Applying P' to the appearance keys and values yields K* = P'K_app and V* = P'V_app, and the final attention is softmax(Q_out(K*)^T / sqrt(d))V*. Because queries define spatial semantics and live in the same embedding space, the authors","pith_inferences":["The matrix S is itself a dense semantic-correspondence map; it could be exported and evaluated directly against ground-truth correspondences, which would test the core assumption more directly than end-to-end image quality.","The hard top-1 selection is the most brittle part: for appearance and structure images with large pose or viewpoint gaps, a top-k aggregation with a small soft window may preserve the sharpness benefit while adding robustness.","The method is applied per self-attention layer, but leakage is likely scale-dependent; stacking query-query rearrangement across multiple layers might give a coarser-to-finer alignment than any single layer.","The proposed vision-language-model scoring protocol could be reused to audit other generative-image tasks where Gram loss and IoU disagree with human perception."],"forward_implications":["Q-Align gives zero-shot appearance transfer without optimization, training, or object masks, so it can be dropped into any pretrained diffusion model's self-attention layers.","Attention leakage is measured at 52.2% of 180 test pairs for the Cross-Image baseline, and Q-Align improves 69.1% of those leaky cases, which should translate to fewer artifacts like background-colored holes or stray object features.","Average appearance-fidelity and structural-consistency scores improve over Cross-Image and other zero-shot baselines across most of the six tested domains: animal faces, animals, cars, birds, buildings, and cakes.","Since the method is layer-local and parameter-free, it can compose with other attention-control editing techniques that also swap or refine keys and values.","The paper's evidence that query and key spaces are incomparable implies other key-value mixing attention methods face the same leakage risk and are candidates for the same fix."],"supporting_citations":[{"why":"Defines cross-image attention for zero-shot appearance transfer and is the baseline whose attention leakage Q-Align targets.","marker":"[1]"},{"why":"Supplies the tuning-free mutual self-attention control mechanism that cross-image attention and Q-Align build upon.","marker":"[3]"},{"why":"Provides the pretrained latent diffusion model whose self-attention layers supply the queries, keys, and values used in the method.","marker":"[28]"},{"why":"Supports the claim that queries and keys reside in distinct embedding spaces, motivating query-query alignment.","marker":"[39]"},{"why":"Mask-based baseline compared against; its fixed-mask composition illustrates the distortion Q-Align claims to avoid.","marker":"[25]"},{"why":"Pre-trained-ViT-feature appearance transfer baseline compared against in the experiments.","marker":"[33]"},{"why":"Diffusion-based image translation baseline compared against in the experiments.","marker":"[18]"},{"why":"Supplies the vision-language model used in the proposed appearance-fidelity and structural-consistency evaluation.","marker":"[15]"},{"why":"Provides the prompting methodology behind the new evaluation protocol, whose scores are aligned with human judgments.","marker":"[22]"}],"fun_headline_variants":["Q-Align: query-query alignment fixes attention leakage","Appearance transfer improved via query-query attention","Zero-shot transfer: query-query alignment stops leaks","Query-query matching prevents attention leakage","Align queries to curb zero-shot appearance transfer leaks"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method assumes that for every position in the structure image, the most similar position in the appearance image is the semantically correct one to copy appearance from; if that similarity ranking is wrong, the transferred colors and textures land on the wrong spots.","fun_headline_variants_meta":{"raw":{"variants":["Q-Align: query-query alignment fixes attention leakage","Appearance transfer improved via query-query attention","Zero-shot transfer: query-query alignment stops leaks","Query-query matching prevents attention leakage","Align queries to curb zero-shot appearance transfer leaks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000431,"raw_usage":{"total_tokens":2005,"prompt_tokens":683,"completion_tokens":1322,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":427,"completion_tokens_details":{"reasoning_tokens":1250}},"tokens_in":427,"tokens_out":1322,"duration_ms":9427,"temperature":1.0,"reasoning_tokens":1250,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:29:24.039839+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take image pairs with ground-truth dense semantic correspondence and compare the accuracy of top-1 query-query matches against top-1 query-key matches; if query-query alignment is not systematically more accurate, the core premise fails. A complementary check: run Q-Align on pairs with extreme pose or category differences and look for outputs where the structure is preserved but the appearance identity lands on semantically wrong regions, which would indicate the alignment matrix is unreliable.","supporting_citations":[{"cited_title":"Cross-image attention for zero-shot appearance transfer","cited_arxiv_id":null,"evidence_quote":"Defines cross-image attention for zero-shot appearance transfer and is the baseline whose attention leakage Q-Align targets."},{"cited_title":"Masactrl: Tuning-free mutual self-attention control for consistent image syn- thesis and editing","cited_arxiv_id":null,"evidence_quote":"Supplies the tuning-free mutual self-attention control mechanism that cross-image attention and Q-Align build upon."},{"cited_title":"Alignment attention by matching key and query distributions","cited_arxiv_id":null,"evidence_quote":"Supports the claim that queries and keys reside in distinct embedding spaces, motivating query-query alignment."},{"cited_title":"Dragondiffu- sion: Enabling drag-style manipulation on diffusion models","cited_arxiv_id":null,"evidence_quote":"Mask-based baseline compared against; its fixed-mask composition illustrates the distortion Q-Align claims to avoid."},{"cited_title":"Splicing vit features for semantic appearance transfer","cited_arxiv_id":null,"evidence_quote":"Pre-trained-ViT-feature appearance transfer baseline compared against in the experiments."},{"cited_title":"Diffusion-based image translation using disentangled style and content representation","cited_arxiv_id":null,"evidence_quote":"Diffusion-based image translation baseline compared against in the experiments."},{"cited_title":"G-eval: NLG evaluation using gpt-4 with better human alignment","cited_arxiv_id":null,"evidence_quote":"Provides the prompting methodology behind the new evaluation protocol, whose scores are aligned with human judgments."}],"review_version":1}