{"id":"376cd374-cd29-4c7a-ad8e-f537b18e8656","arxiv_id":"2608.13037","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GATO-Vid derives a closed-form query-steering direction for cross-attention logits and injects it into early DiT blocks, achieving IoU 0.363 against 0.249 for the best baseline on a 400-video grounded-generation benchmark.","lead":"This paper introduces GATO-Vid, which places objects at user-specified locations in generated videos by computing a cross-attention steering direction analytically instead of backpropagating through the model. On the Wan2.2 backbone it reports roughly 2.4 times the IoU of the best gradient-free baseline at 0.4% added inference time, with a measured drop in motion and aesthetics.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's negative-bias direction contradicts the derived optimum, so the reported gains may not test the analytical mechanism claimed.","rationale":"The reader's weakest_assumption is essentially the same concern: the analytically derived optimum is not what Algorithm 1 injects, and the real-model validation is only a toy demonstration. I agree that this is the most load-bearing issue. The empirical claim of large localization gains is plausible, and the 0.4% overhead claim is well-supported by the experimental setup. However, the paper's contribution is explicitly framed as 'a theoretical foundation that optimizes a surrogate score on the fly' and 'an exact, closed-form solution.' If the implemented negative bias differs from that solution, the reported gains could come from a different heuristic, and the analytical narrative would not explain the results. Ablation (iv) shows that the background term matters, so this is not a cosmetic discrepancy. A controlled comparison of the two negative-bias directions would settle whether the derived optimum is actually the source of the improvement. Until then, the paper should be conditional: the empirical result may stand, but the method description and the theoretical claim must be aligned, and the code should be checked against Eq. (16).","tokens_in":16398,"tokens_out":4240,"duration_ms":49142,"concrete_test":"Run the Set 1 and Set 2 evaluations with the negative bias corrected to b- = -K(T)/||K(T)|| as in Eq. (13), keeping all other hyperparameters (λ, Gaussian modulation, block/iteration schedule) identical to Algorithm 1. If the IoU/CD numbers move materially outside the reported margins, the method as evaluated is not the analytically derived method; if they match, Algorithm 1 is a typo and the derived optimum is validated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central scientific claim is that the closed-form optimum of the surrogate score in Eq. (10) yields the query-steering directions that produce the Table 1 gains. But Algorithm 1 does not implement the derived negative direction. Eq. (13) states b- = -K(T)/||K(T)||, which suppresses the target text tokens outside the mask. Algorithm 1 instead sets b- = Normalize(mean(k[Tc])) and applies q' <- q' - λ+||q'||b-, i.e. it suppresses the complement text tokens in background regions, a different mechanism. It also uses λ+ rather than λ- and omits the orthogonal projection of Eq. (16). The ablation in Table 2 shows the negative component is load-bearing: removing the background bias drops IoU from 0.363 to 0.241 on Set 1. Thus either the derivation does not explain the implemented algorithm, or Algorithm 1 is not what the code actually runs. If the latter, the paper's theoretical narrative is disconnected from the evaluated method. Separately, the toy validation in Fig. 1b is on a random 3-layer attention model and shows only that the injection reduces an l2 loss in that setting; it does not establish that the analytically optimal direction survives RMSNorm re-projection, cross-attention softmax, and 40-block DiT interactions. The empirical headline may still hold, but the mechanism claimed is not the mechanism tested as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GATO-Vid, a training-free and gradient-free method for spatially grounded text-to-video generation that steers cross-attention in a DiT-based video generator (Wan2.2). The authors introduce a surrogate score over pre-softmax logits (Eq. 9) that factorizes into three dot products, solve for the analytically optimal query directions (Eq. 13), and inject these directions into query tokens with an RMSNorm-aware renormalization (Eq. 16). On two new evaluation sets (400 videos each), GATO-Vid reports substantially higher IoU and lower Center Distance than gradient-free baselines Peekaboo, VideoTetris, and SwitchCraft, with only 0.4% added inference time, at the cost of some dynamic degree and aesthetic quality. The paper also presents ablations showing that the positive and negative bias components, the ellipsoid projection, and the Gaussian modulation each contribute to the reported localization gains.","tokens_in":16681,"tokens_out":6957,"duration_ms":67370,"significance":"If the claims hold, GATO-Vid would be a practical step: it avoids backpropagation through a large DiT, keeps inference overhead near zero, and achieves markedly better localization than existing training-free gradient-free alternatives. The analytical derivation is self-contained and verifiable, the experimental design includes a new benchmark and an ablation study, and the authors state that code will be released. However, the central theoretical narrative is not matched by the implemented algorithm (Algorithm 1 deviates from Eq. 13 and Eq. 16), and the evidence for the mechanism is limited to a toy model. The lack of statistical significance reporting weakens the empirical claim. These issues require major revision rather than acceptance.","major_comments":[{"comment":"There is a direct contradiction between the derived optimal direction and the implemented negative bias. Equation (13) defines b^- = -K(T)/||K(T)||, whose action is to suppress, in the background region, the attention to target text tokens; Algorithm 1, however, defines b^- = Normalize(mean(k[Tc])) and injects q' ← q' - λ+||q'||b^-, which suppresses, in the background, the attention to complement text tokens. The pseudocode also uses λ+ for both injections and omits the orthogonal projection of Eq. (16). Because Table 2 shows that removing the background component decreases Set-1 IoU from 0.363 to 0.241, this component is load-bearing; as written, the mechanism that produces the reported gains is not the mechanism derived in Sec. 3.2. The authors should either implement the derived b^- (and the projection) or revise the theoretical claim and add experiments that identify which direction actually drives the improvement.","section":"§3.3, Algorithm 1 vs. Eq. (13)"},{"comment":"The toy experiment validates the surrogate score on a random three-layer cross-attention model, but it does not validate that the analytically optimal direction survives the full injection pipeline in a 40-block DiT, including RMSNorm re-projection, softmax, and cross-attention interactions. Since the paper's central claim is that the closed-form optimum of Eq. (10) yields the effective steering direction, the authors should provide evidence that the score s in Eq. (9) actually increases after injection in Wan2.2 (e.g., a block-wise diagnostic of attention scores), or temper the claim to state that the empirical gains are observed for the implemented algorithm rather than for the derived optimum.","section":"§3.3, Fig. 1b"},{"comment":"The central empirical claim that GATO-Vid 'significantly outperforms' existing baselines is supported only by point estimates with no error bars, confidence intervals, or significance tests. Given the high variance of text-to-video generation and the use of four seeds per prompt, the reader cannot determine whether the reported margins (e.g., Set-1 CD 0.059 vs. 0.103) are robust. The authors should report standard errors or confidence intervals, and ideally a paired significance test across the shared random seeds, to substantiate the claim of significance.","section":"§4.2, Table 1"}],"minor_comments":[{"comment":"There are several typos and spacing errors, e.g., 'this a task' in the second paragraph and 'lessthan32GBavailableVRAM' in the first paragraph; the text should be carefully proofread.","section":"§1, Introduction"},{"comment":"The display of Eq. (14) is difficult to parse due to missing parentheses and ambiguous placement of the denominator; please rewrite with a clear fraction notation for the normalization term.","section":"§3.3, Eq. (14)"},{"comment":"The notation for the hidden dimension is inconsistent: the body of the paper uses d, while Algorithm 1 and parts of Sec. 4.1 use n; please unify the notation.","section":"§4.1, Implementation Details"},{"comment":"The hyperparameter λ± is described as 'set to 1.5 with a linear decay,' but the decay schedule (over iterations? to what final value?) is not specified; please clarify how λ+ and λ- are computed at each step.","section":"§4.1, Implementation Details"},{"comment":"The CD and IoU metrics are computed only on videos with successful SAM 3 detections, which introduces a selection bias; please report the number of videos used for each metric or discuss the potential bias.","section":"§4.2, Evaluation Metrics"},{"comment":"The final sentence of the conclusion contains a doubled 'while' ('while our experiments suggest that while stronger spatial control...'); this should be corrected.","section":"§5, Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The equation-to-algorithm mismatch is the main risk to the paper's central claim. If the authors cannot reconcile the derived optimum with the implemented negative bias, the theoretical narrative would not be supported by the reported experiments. Additionally, the paper depends on several 2026 citations (e.g., SAM 3, SwitchCraft, Gemini) that may not be publicly available at the time of review; please verify with the authors that all referenced resources are accessible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I'll get straight to it: this paper has a real empirical result and a real problem. The result is that a simple, gradient-free query-steering method gets IoU 0.363 on their Set 1 against Peekaboo's 0.249, with only 0.4% added inference time on Wan2.2. If that holds up, it's a useful tool for people who need spatial control on modern DiT video models without multiple GPUs. The closed-form surrogate score is a neat idea: instead of backpropping an attention loss, they separate the logit terms and solve for the optimal query direction. That derivation is correct as far as it goes.\n\nThe problem is that Algorithm 1 implements something different from the derived optimum. Eq. (13) says the negative background bias should be -K(T)/||K(T)||, i.e., antiparallel to the mean target text key. Algorithm 1 sets b- = Normalize(mean(k[Tc])) and then subtracts it from background queries. That is suppressing the complement text tokens outside the mask, not the target tokens. It also uses λ+ for both injections and omits the orthogonal projection from Eq. (16). The ablation says removing the negative bias drops IoU from 0.363 to 0.241, so this component is load-bearing. But if the tested component is the wrong direction, the empirical gains don't validate the theory. This is not a minor typo; it breaks the equation-to-code link.\n\nThere are other soft spots. No error bars or significance tests on the main quantitative claims. Single backbone. The toy validation is on a random 3-layer model and only shows the surrogate score correlates with l2 loss, not that the injection survives RMSNorm, softmax, and 40-block interactions. The hyperparameter search for baselines is described but not detailed.\n\nThat said, the paper is worth engaging. The method is practical, the overhead claim is striking, and the analytical surrogate is a genuinely different angle from the gradient-based and masking baselines. The fix is clear: align the pseudocode with the derivation (or rewrite the derivation to match what the code actually does), add error bars, and run the toy experiment on the actual architecture with the actual injection. A serious referee should ask for those changes, but not desk-reject.\n\nWho is this for? Anyone working on training-free control of DiT video generators. It will be more useful after the inconsistency is resolved.","headline":"GATO-Vid reports large IoU gains for gradient-free spatial grounding on Wan2.2, but the pseudocode does not match the derived optimum, so the mechanism behind the gains is unclear as written.","tokens_in":17187,"tokens_out":5106,"would_cite":false,"duration_ms":47498,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GATO-Vid shows that spatial grounding in text-to-video generation can be achieved by a closed-form update to cross-attention query directions, with reported IoU gains from 0.154 to 0.363 on one benchmark at 0.4% added inference time.","keywords":["text-to-video generation","spatial grounding","training-free control","gradient-free optimization","cross-attention guidance","diffusion transformer","query injection","flow matching"],"falsifier":"On Wan2.2, compare three variants on Set 1: the implemented Algorithm 1, the same algorithm with the negative bias replaced by the derived $-K(T)/\\|K(T)\\|$, and the same with the RMS re-projection removed. If the derived-direction variant does not match or beat the reported 0.363 IoU, the paper's stated mechanism (the analytical surrogate) is not what the empirical gains test.","tokens_in":16192,"feed_emoji":"📍","tokens_out":10564,"duration_ms":104198,"temperature":0.7,"pith_summary":"GATO-Vid is an attempt to make spatial grounding in text-to-video generation practical. Where earlier training-free methods need backpropagation through billions of parameters to adjust object locations, GATO-Vid derives the optimal direction for a simplified cross-attention score in closed form and injects that direction into the model's query vectors during sampling. The paper reports that this raises object-box IoU from 0.154 to 0.363 on its first benchmark, against 0.249 for the best existing gradient-free baseline, while adding 0.4% inference time. It also reports a quality trade-off: better localization lowers scene dynamics and aesthetic scores, so the method is best seen as a steering tool whose injection schedule can be tuned.","feed_headline":"One closed-form direction places video objects where you ask","feed_subtitle":"GATO-Vid beats gradient-free baselines on object-box accuracy while adding only 0.4% inference time.","key_machinery":"The central object is the surrogate score $s = Q(M)\\cdot\\big(K(T)-K(T^c)\\big) - Q(M^c)\\cdot K(T)$, where $Q(M)$ and $Q(M^c)$ are mean query vectors inside and outside the target mask and $K(T)$, $K(T^c)$ are mean key vectors of the target and non-target text tokens. Its closed-form maximizers are $b_+ = (K(T)-K(T^c))/\\|K(T)-K(T^c)\\|$ for masked queries and $b_- = -K(T)/\\|K(T)\\|$ for background queries. The injection mechanism carries these directions into each early transformer block: edited queries become $q_j + \\lambda_+ \\|q_j\\| b_+$ inside the mask and $q_j + \\lambda_- P(b_-\\|q_j\\|, q_j)$ outside, with $P$ the orthogonal projection that strips the component parallel to the original query, followed by RMSNorm-style re-scaling with the block's gain vector so the edited query stays on the hyper-ellipsoid. A Gaussian-shaped $\\lambda_+$ across the bounding box keeps the attention peak at the object center rather than flattening it.","core_discovery":"The paper's central claim is that gradient-based attention guidance can be replaced by an exact, analytically computed update to query vectors. Starting from the pre-softmax logits, it defines a score with three terms: raise the logits between target-region queries and target-word keys, lower the logits between target-region queries and non-target keys, and lower the logits between non-target queries and target keys. Because this score is a sum of separable inner products, its maximizer is explicit: inside the mask the query mean should move toward $K(T)-K(T^c)$, and outside the mask it should move against $K(T)$, where $K(T)$ denotes the mean key of the target text tokens. The paper's injection scheme adds these directions to the per-query vectors, keeps the background correction orthogonal to the original query, and re-projects onto the RMSNorm ellipsoid so the edits remain within the latent manifold the network expects. On Wan2.2, with two 400-video benchmarks and four seeds per prompt, the paper reports consistent gains on both IoU and center distance over Peekaboo, VideoTetris, and SwitchCraft, with a 0.4% inference-time overhead.","pith_inferences":["We infer that the implemented negative bias $\\mathrm{Normalize}(\\mathrm{mean}(k[T_c]))$ is not the derived optimum $-K(T)/\\|K(T)\\|$, so a head-to-head comparison would separate the analytical component from the empirical injection recipe.","We infer that the same closed-form score and injection rule should transfer to any DiT with cross-attention and RMSNorm, including image generation, since the derivation uses only masks and mean keys, not video-specific structure.","We infer that Fig. 1b's monotonicity result on a random three-layer model is the weakest empirical anchor; reproducing it on the first 20 blocks of Wan2.2 with real prompts would show whether the surrogate score, not the re-normalization, drives the reported gains.","We infer that an adaptive injection policy—applying the bias only when early-block attention scores are high, per Fig. 4c—could keep the localization gains while recovering some of the lost dynamics and aesthetic quality."],"forward_implications":["Object localization on Wan2.2 no longer requires a backward pass: a single backprop step costs +300% inference time and 26 GB extra VRAM, while GATO-Vid adds 0.4%.","Steering the first 20 transformer blocks during the first 15% of denoising steps is as effective as steering all 40 blocks, so spatial layout is decided early in the network.","Removing the negative bias (suppressing target tokens outside the mask) degrades localization more than removing the positive bias, making the negative term the main driver of grounding.","Localization and quality are a tunable trade-off: increasing the number of guided iterations raises IoU but lowers aesthetic quality and dynamics."],"supporting_citations":[{"why":"The Wan2.2 DiT generator on which GATO-Vid and all compared baselines are implemented and evaluated.","marker":"[39]"},{"why":"Peekaboo is the strongest gradient-free baseline the method competes with on localization (best IoU among baselines).","marker":"[21]"},{"why":"SwitchCraft is the closest prior query-steering method on the Wan2.x architecture and a key baseline.","marker":"[48]"},{"why":"VideoTetris is a compositional gradient-free baseline re-implemented on Wan2.2 for comparison.","marker":"[38]"},{"why":"It grounds the premise that cross-attention layers localize objects, motivating query-key manipulation as the control channel.","marker":"[34]"},{"why":"It defines RMSNorm, whose hyper-ellipsoid geometry the query re-projection in Eq. (16) preserves.","marker":"[53]"},{"why":"It defines the Diffusion Transformer block structure used by the target model.","marker":"[30]"},{"why":"It supplies the flow-matching sampling framework within which the latent update is performed.","marker":"[24]"}],"fun_headline_variants":["Gradient-free analytic guidance puts video objects exactly where you ask","No gradients, no training: closed-form step places video objects precisely","Analytic closed-form update steers video objects without backprop","Exact gradient-free optimization places objects in video with tiny overhead","One analytic step, zero gradients: place video objects precisely"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the direction maximizing the simplified logit score, once added to the model's queries and re-normalized, actually moves objects toward the target box inside the full 40-block video transformer; the paper demonstrates this monotone link on a random three-layer toy attention model, not on the real network.","fun_headline_variants_meta":{"raw":{"variants":["Gradient-free analytic guidance puts video objects exactly where you ask","No gradients, no training: closed-form step places video objects precisely","Analytic closed-form update steers video objects without backprop","Exact gradient-free optimization places objects in video with tiny overhead","One analytic step, zero gradients: place video objects precisely"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00115,"raw_usage":{"total_tokens":4771,"prompt_tokens":953,"completion_tokens":3818,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":3732}},"tokens_in":569,"tokens_out":3818,"duration_ms":28995,"temperature":1.0,"reasoning_tokens":3732,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:04:14.761075+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On Wan2.2, compare three variants on Set 1: the implemented Algorithm 1, the same algorithm with the negative bias replaced by the derived $-K(T)/\\|K(T)\\|$, and the same with the RMS re-projection removed. If the derived-direction variant does not match or beat the reported 0.363 IoU, the paper's stated mechanism (the analytical surrogate) is not what the empirical gains test.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","cited_arxiv_id":null,"evidence_quote":"Peekaboo is the strongest gradient-free baseline the method competes with on localization (best IoU among baselines)."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference 20 G","cited_arxiv_id":null,"evidence_quote":"SwitchCraft is the closest prior query-steering method on the Wan2.x architecture and a key baseline."},{"cited_title":"In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)","cited_arxiv_id":null,"evidence_quote":"It grounds the premise that cross-attention layers localize objects, motivating query-key manipulation as the control channel."},{"cited_title":"In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R","cited_arxiv_id":null,"evidence_quote":"It defines RMSNorm, whose hyper-ellipsoid geometry the query re-projection in Eq. (16) preserves."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It defines the Diffusion Transformer block structure used by the target model."}],"review_version":1}