{"id":"1c0f6ea6-79de-4b0c-9d75-849b12546daa","arxiv_id":"2609.00161","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"IMPACT converts object-conditioned cross-attention into a training-weighting map that corrects the supervision-allocation mismatch in MSE-based world model training, improving interaction fidelity on robot arm and human hand tasks.","lead":"IMPACT reweights the training loss of video world models using the model's own cross-attention maps, focusing supervision on interaction regions without external annotations. This makes interaction-aware world model training scalable and eliminates the need for costly auxiliary motion or geometry estimators.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Early-training local prediction errors ℓ_p may not reliably distinguish interaction regions from static content, as cross-attention maps are diffuse and ℓ_p is uniformly high, but the paper does not analyze this regime.","rationale":"The paper's strongest evidence is the significant quantitative improvement across two benchmarks, which suggests that the overall method works in practice. The reader identified the same weakest assumption, and the concern is genuine, but the experimental results already imply that ℓ_p is sufficiently discriminative in the trained regime. The lack of early-training analysis is a documentation gap rather than a fatal flaw; the central claim is still well-supported by the final performance. Therefore, the stress-test does not change the ACCEPT verdict, but the concrete test would strengthen the paper's argument and resolve the identified weakness.","tokens_in":21952,"tokens_out":4398,"duration_ms":41026,"concrete_test":"Compute the Spearman rank correlation between per-position ℓ_p (detached, as used in Eq. 13) and a ground-truth interaction signal (e.g., optical flow magnitude thresholded at the 90th percentile) on a held-out set of 100 robot-arm training clips, at initialization (step 0) and after 100, 500, 1000, and 5000 training steps. Also compute the average intersection-over-union (IoU) between the interaction map G (binarized at 0.5) and the motion mask. If the correlation is below 0.3 or the IoU below 0.4 at step 0, then ℓ_p is not discriminative in the early training regime that ADS depends on. A correlation below 0.3 after 500 steps would indicate that the calibration remains unreliable for a significant period, undermining the claim that ADS consistently improves over the raw attention prior.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The ADS calibration (Eq. 13–14) relies on ℓ_p (Eq. 3) being higher in dynamic interaction regions than in static regions. Early in fine-tuning, the pretrained backbone (Wan 2.2 / Cosmos) may produce low ℓ_p on static backgrounds but also diffuse cross-attention maps A that spread probability across the entire scene (Fig. 4 illustrates a diffuse A). Candidates M_k are sampled from this diffuse prior; their scores d_k are the average ℓ_p within each mask. If ℓ_p is not yet discriminative (e.g., the model's velocity predictions are noisy everywhere), the weights α_k become near-uniform and the interaction map G reduces to the unweighted average of candidates, eliminating the calibration benefit. The paper provides no analysis of the discriminability of ℓ_p at initialization or during early training, nor does it quantify the correlation between ℓ_p and true interaction motion (e.g., optical flow). This is the weakest link because the entire ADS mechanism hinges on ℓ_p providing a reliable signal; if it fails early, the method may rely entirely on the raw attention prior, and the claimed +0.92 EWMScore gain from ADS (Table 3) may not materialize in practice when training from scratch or with a different backbone. The reader's weakest_assumption correctly identifies this gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper identifies a supervision-allocation mismatch in world model training under the standard global MSE denoising objective: static content dominates, leaving dynamic interaction regions under-supervised. To address this, the authors propose IMPACT, a training framework that uses object-conditioned cross-attention from the model's own forward pass as a spatiotemporal prior for interaction regions. The framework has two components: Attention Distribution Sampling (ADS) calibrates cross-attention proposals by weighting sampled candidate regions with detached local prediction errors to form an interaction map, and Interaction-Weighted Supervision (IWS) uses this map to reweight the denoising loss, with gradient routing to prevent the cross-attention parameters from adapting to the reweighted objective. Experiments on robot-arm manipulation (WorldArena) and human-hand manipulation (EgoDex) with two DiT backbones (Wan 2.2, Cosmos-Predict 2.5) show that IMPACT consistently improves interaction fidelity, physical plausibility, and visual quality over uniform MSE training and several baselines.","tokens_in":22425,"tokens_out":2984,"duration_ms":28864,"significance":"If the claimed improvements hold, IMPACT offers a practical and scalable way to improve interaction quality in world models without requiring external dense representations (optical flow, depth, etc.) or inference-time modifications. The core idea of repurposing the model's own cross-attention as a prior and calibrating it with prediction errors is novel and well-motivated. The empirical gains are substantial: for example, the Wan 2.2-AC + IMPACT variant raises EWMScore by 3.81 points on WorldArena and reduces FVD from 366 to 111 on EgoDex. The ablation study (Table 3) confirms that both ADS and IWS contribute. The manuscript is clearly written and the method is reproducible from the provided equations.","major_comments":[{"comment":"The ADS calibration relies on local prediction errors ℓ_p being higher in dynamic interaction regions than in static regions. During early fine-tuning steps, the pretrained backbone may produce ℓ_p that is high across all regions, making the candidate scores d_k near-uniform and the weights α_k uninformative. The paper does not analyze the discriminability of ℓ_p at initialization or during early training, nor does it quantify the correlation between ℓ_p and ground-truth interaction motion (e.g., optical flow). This gap threatens the central claim that ADS provides a meaningful calibration benefit; the reader's weakest_assumption correctly identifies this issue. The authors should provide an analysis (e.g., plots of ℓ_p variance over training steps, or an ablation where ADS is replaced with uniform weights at early steps) to justify that the calibration signal is reliable from the start.","section":"§3.2, Eq. (13)"},{"comment":"Object-phrase extraction uses a frozen Qwen2.5-0.5B model, but the paper reports no quantitative evaluation of its accuracy or failure rate for the domains tested. If the extraction incorrectly identifies the manipulated object (e.g., picks a background element or misses the object), the entire cross-attention prior is misaligned and the interaction map may target irrelevant regions. The authors should report extraction accuracy on a manually labeled subset of the training data (e.g., 100 samples per domain) or at least discuss observed failure modes and their impact.","section":"§3.1, §A.3"},{"comment":"The method introduces eight free hyperparameters (K, σ, ρ, κ, β, γ, τ, coarse grid size) that are fixed uniformly across both domains and backbones. The paper provides no sensitivity analysis for any of them. For instance, γ controls the maximum regional emphasis and could critically affect the trade-off between interaction and background supervision. Without ablations (e.g., varying γ from 2 to 10) or a justification that the results are robust to moderate changes, the reader cannot assess how carefully these values must be tuned. At minimum, the authors should include a sensitivity study for γ and K.","section":"§3.2–3.3, hyperparameters"},{"comment":"The Action Following score for Cosmos-Predict 2.5 (action) jumps from 0.0133 to 0.6260 with IMPACT. This is an enormous relative change (47×) that far exceeds any other metric improvment in the table. The metric definition (Table 6) measures response diversity under three different instructions sharing one initial frame. Such a drastic increase may indicate an artifact (e.g., the generated videos becoming more diverse but perhaps less faithful to the action sequence). The authors should provide an explanation and sanity-check the metric (e.g., by showing example triplets or noting whether the raw dissimilarity values are plausible). Without clarification, this improvement appears questionable.","section":"Table 8, row Cosmos + IMPACT"},{"comment":"The paper compares IMPACT against many baselines, but several key ones (e.g.\n, WoW, CtrlWorld) are tested under different inference settings or backbone sizes (e.g., Wan 2.6 vs Wan 2.2). The EWMScore table (Table 1) shows IMPACT on Wan 2.2-AC surpassing Wan 2.6 by 0.6 points. While this is impressive, the comparison is not entirely apple-to-apple because Wan 2.6 is a larger model. The authors should clearly state the model capacity differences and discuss whether the improvements are attributable to the training procedure rather than to backbone architecture.","section":"§4.1, baseline comparison"}],"minor_comments":[{"comment":"Typo: 'supervision-allocation mismatchunder the globally averaged' missing space before 'under'.","section":"Abstract"},{"comment":"The notation 'sg[A]' should be explained earlier; it is formally introduced only later in the text. Consider adding a brief note when first used.","section":"§3.2, Eq. (9)"},{"comment":"The table caption says models are grouped into general, embodied, and representation-guided, but the actual grouping in the table omits the 'Embodied' and 'Representation-guided' labels—only 'Embodied world models' and 'Representation-guided models' lines are there. Add visual grouping lines for clarity.","section":"Table 1, footnote"},{"comment":"The color bar in Figure 4 is described as 'light-to-dark' but the actual map uses a jet-like colormap. Use a more intuitive colormap (e.g., yellow-to-red) and indicate that the bar encodes the candidate error d_k, not the final weight α_k.","section":"Figure 4"},{"comment":"The row '+ IWS (IMPACT) in Table 3 shows an EWMScore of 61.54, but from the ablation discussion it seems this row corresponds to 'Wan 2.2-AC + IWS' without ADS. Label it clearly as 'Wan 2.2-AC + IWS' to match the main text.","section":"Table 3"},{"comment":"The annotation prompt returns JSON with a list of objects. If the instruction lists multiple manipulated objects (e.g., 'Pick up the red block and the blue block'), does the extraction handle multiple entries? The paper should clarify.","section":"Appendix A.3"}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"The paper proposes IMPACT, a training framework that reweights the denoising MSE loss toward interaction regions using the model's own cross-attention maps, calibrated by local prediction errors. The core idea is straightforward: use attention as a coarse prior, sample candidate regions, score them with detached prediction errors, and form a soft interaction map that upweights dynamic content. No external optical flow or depth needed.\n\nWhat's new and well done: the combination of attention sampling with prediction-error calibration is genuinely novel—it's not just another loss weighting trick. The authors identify a real supervision-allocation problem and address it elegantly. Experiments are thorough: two backbones (Wan 2.2, Cosmos), two domains (robot arm, human hand), multiple baselines, and a clean ablation. The gains are substantial and consistent. The gradient routing design (Eq. 18–19) correctly prevents the attention-producing parameters from being polluted by the interaction-weighted loss, which addresses the circularity concern.\n\nSoft spots: (1) The object-phrase extraction uses a frozen 0.5B Qwen model. No accuracy numbers are reported for this step. If the extraction fails on complex or ambiguous instructions, the whole pipeline degrades. (2) The stress-test point about early-training ℓ_p is legitimate—early in fine-tuning, prediction errors may be high everywhere and not discriminative. The paper shows one visualization of a working forward pass (Fig. 4) but no analysis of how ℓ_p discriminability evolves over training steps. This is the weakest link. However, the ablation shows that even without ADS (just using raw attention), IWS alone gives +2.89 EWMScore, so the method does not collapse even if the calibration is imperfect early on. (3) The paper uses private data (RoboTwin, EgoDex) and does not release code, limiting reproducibility.\n\nOverall: this is a solid, well-motivated paper with convincing experiments. The concerns are real but minor in impact. I would send this to peer review—the method is novel enough and the results strong enough to warrant scrutiny. I wouldn't cite it directly in my own work (different subarea), but I'd recommend it to anyone working on interactive world models.\n\nRecommendation: accept with minor revisions. Ask for object-extraction accuracy and an analysis of ℓ_p discriminability over training.","headline":"Clean method for reweighting world-model supervision using cross-attention + prediction errors, with strong experiments; the early-training robustness concern is real but not fatal.","tokens_in":22928,"tokens_out":901,"duration_ms":10560},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"Reweighting a world model's denoising objective by its own cross-attention, calibrated with prediction errors, targets sparse interaction regions and improves generated interactions without external dense signals or inference changes.","keywords":["interaction-aware world models","supervision-allocation mismatch","cross-attention prior","attention distribution sampling","interaction-weighted supervision","flow matching","latent video diffusion","manipulation world models"],"falsifier":"Take an early training checkpoint, for instance ten percent through the single epoch, and measure whether the ADS candidate scores $d_k$ rank true interaction regions, meaning the gripper–object or hand–object contacts visible in the dataset annotations, above static regions; if that rank correlation is near zero while the attention mass is already on the object, the calibration stage adds no signal and any gain must come from the reweighting alone. A companion check is to retrain with the weighting concentration $β$ set to zero so candidates are averaged uniformly, and with $γ = 1$ so no reweighting occurs, to see whether the reported improvement gap persists.","tokens_in":23753,"feed_emoji":"🎬","tokens_out":9371,"duration_ms":91106,"temperature":0.7,"pith_summary":"World models trained to predict what happens next under a commanded action inherit a supervision-allocation mismatch: the standard flow-matching objective averages the denoising error uniformly over all spatiotemporal positions, so abundant static pixels dominate the gradient while the small regions where a robot gripper or a human hand actually changes the scene stay under-supervised. IMPACT is a training-only framework that corrects this mismatch by treating the diffusion transformer's own cross-attention on the manipulated-object words as a prior for where action-conditioned change occurs. It samples candidate regions from that attention prior, scores each candidate by the mean local prediction error inside it, and uses the resulting interaction map to reweight the denoising loss toward interaction regions, with gradient routing so the attention-producing parameters keep following the global objective. The paper claims this consistently beats uniform-MSE training on both robot-arm and human-hand manipulation across two diffusion-transformer backbones and two control modalities, improving interaction fidelity, physical plausibility, and visual quality without any optical flow, depth, 3D, or annotation signal. If right, it would make interaction-aware world-model training as scalable as ordinary video-generation training.","feed_headline":"Sharpen interactions in world-model video with one training change","feed_subtitle":"The model's own cross-attention, calibrated by prediction errors, targets sparse contact regions with no extra supervision needed.","key_machinery":"The load-bearing object is the interaction map $G$ constructed by Attention Distribution Sampling: a soft spatiotemporal weight field built in one training step from the object-conditioned cross-attention map $A$, which is the average over selected blocks and heads of the attention probability mass assigned to the manipulated-object tokens. ADS samples $K = 8$ equal-area candidate masks from a tempered, noise-perturbed version of $A$, scores each by its mean detached local prediction error, converts the scores into softmax weights, and aggregates the weighted candidates into $G$. Interaction-Weighted Supervision then uses $G$ to reweight the denoising loss per position, keeping the global objective and normalizing by total weight so the loss scale does not grow, while gradient routing runs two backward passes so the attention-producing parameters continue following the original uniform objective and the remaining parameters follow the interaction-weighted one. Every operation in ADS and IWS is training-only, so the model at inference has the same architecture and sampling procedure as the MSE-trained backbone.","core_discovery":"The central claim is that the supervision-allocation mismatch of the globally averaged MSE denoising objective, rather than a lack of interaction knowledge in the backbone, is what makes world models mangle contact regions, and that the model's own attention can close the gap. In IMPACT's forward pass, the manipulated-object text tokens are grounded from the instruction alone by a frozen language model; their cross-attention mass $A$ is averaged over selected heads and blocks, tempered, converted to logits, and perturbed with coarse-grid Gaussian noise to sample $K = 8$ fixed-area candidate masks. Each candidate is scored by its mean detached local prediction error $d_k$, turned into softmax weights $α_k$, and aggregated into an interaction map $G$. The backward pass reweights every position by $w_p = 1 + (γ - 1)G_p$ with $γ = 5$, so interaction positions receive up to five times the denoising supervision while the total loss stays normalized and the whole field keeps supervision; gradient routing gives the cross-attention parameters the global objective and the remaining transformer parameters the interaction-weighted one. The paper reports that on WorldArena this raises the overall EWMScore from 58.65 to 62.46 on Wan 2.2-AC and from 55.91 to 62.53 on Cosmos-Predict 2.5 action, and on EgoDex cuts FVD from 366.12 to 110.94 and FID from 44.71 to 5.79 while improving the hand-interaction metrics, with the component ablation attributing +2.89 points to the reweighting and a further +0.92 to the attention calibration.","pith_inferences":["The supervision-allocation argument extends beyond manipulation: any action-conditioned video model dominated by static content, such as driving scenes conditioned on an ego trajectory or navigation conditioned on a language goal, could plausibly benefit from the same attention-as-interaction-map reweighting.","Object grounding is instruction-only and inherited from a frozen language model, so instructions that name no manipulable object, or refer to it by a pronoun such as 'it', would leave the attention anchor empty and reduce the interaction map to prediction-error weighting alone; measuring that degradation is a natural stress test the paper does not run.","On the Cosmos backbone the paper notes that gains concentrate in interaction quality and action following while several other dimensions decline, which suggests the emphasis strength $γ$ may need to be sample-dependent, for instance gated by how concentrated the object-token attention mass is, a testable extension the paper leaves implicit.","The calibration scheme stops gradients through both the attention map and the candidate scores, so the model cannot shrink the interaction region to reduce the weighted loss; a further check on whether that detachment is necessary would be to allow gradients into the candidate weights and compare stability."],"forward_implications":["World-model training gains the scalability of plain video generation: the interaction signal comes from the standard forward pass, so no auxiliary estimators, dense annotations, or preprocessing pipelines are needed no matter how large the corpus grows.","Inference is untouched: IMPACT changes only the loss, so the deployed model keeps the backbone's architecture, conditioning interface, and sampling schedule.","The gains are not tied to one backbone or control modality, since the paper reports consistent improvements on a trajectory-conditioned robot-arm setting and a pose-conditioned human-hand setting across two different diffusion-transformer backbones.","The two components are complementary: the reweighting alone supplies most of the improvement, with the prediction-error calibration of the attention prior adding a further gain, implying that raw cross-attention is a useful but imperfect interaction map.","Because the supervision is reallocated rather than masked, regions outside the estimated interaction map keep their original unit weight, so the approach preserves global training signal while concentrating it on sparse dynamic content."],"supporting_citations":[{"why":"Supplies the flow-matching denoising objective that IMPACT reweights with its interaction map.","marker":"Lipman et al. 2023"},{"why":"Provides the Wan 2.2 diffusion-transformer backbone whose cross-attention is harvested as the interaction prior.","marker":"Wan Team et al. 2025"},{"why":"Provides the Cosmos-Predict 2.5 backbone and its action-conditioned variant, the second model IMPACT is applied to and compared against.","marker":"NVIDIA et al. 2025"},{"why":"Frozen Qwen2.5-0.5B extracts the manipulated-object noun phrase from the instruction for object-token grounding.","marker":"Qwen et al. 2025"},{"why":"WorldArena benchmark supplies the 16 normalized metrics and overall EWMScore used to evaluate robot-arm generation quality.","marker":"Shang et al. 2026"},{"why":"EgoDex supplies the egocentric human-hand manipulation clips, hand-pose tracks, and test set used in the hand setting.","marker":"Hoque et al. 2025"},{"why":"RoboTwin 2.0 generates the simulated bimanual rollouts used as the robot-arm training corpus.","marker":"Chen et al. 2025"}],"fun_headline_variants":["Attention maps fix world-model contact physics without extra labels","Train world models to handle interactions by reweighting loss","Use cross-attention prior to boost interaction fidelity in video","Cut FVD 70% in hand manipulation with attention calibration","No extra supervision needed: attention calibrates world model physics"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method assumes that the mean local prediction error inside a sampled candidate region reliably indicates where the interaction is happening at the moment of training, but early in training the error is large in nearly every region, so it may not yet distinguish the manipulation site from static content, and the paper does not analyze how sensitive the interaction map is to the noise level of those early-training errors.","fun_headline_variants_meta":{"raw":{"variants":["Attention maps fix world-model contact physics without extra labels","Train world models to handle interactions by reweighting loss","Use cross-attention prior to boost interaction fidelity in video","Cut FVD 70% in hand manipulation with attention calibration","No extra supervision needed: attention calibrates world model physics"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000454,"raw_usage":{"cost":0.000453768,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000453768,"upstream_inference_prompt_cost":0.000129808,"upstream_inference_completions_cost":0.00032396},"total_tokens":2289,"prompt_tokens":1132,"completion_tokens":1157,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":1075}},"tokens_in":1132,"tokens_out":1157,"cost_source":"openrouter_usage","duration_ms":11298,"temperature":1.0,"reasoning_tokens":1075,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-09-09T02:41:32.401266+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-alltime-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Take an early training checkpoint, for instance ten percent through the single epoch, and measure whether the ADS candidate scores $d_k$ rank true interaction regions, meaning the gripper–object or hand–object contacts visible in the dataset annotations, above static regions; if that rank correlation is near zero while the attention mass is already on the object, the calibration stage adds no signal and any gain must come from the reweighting alone. A companion check is to retrain with the weighting concentration $β$ set to zero so candidates are averaged uniformly, and with $γ = 1$ so no reweighting occurs, to see whether the reported improvement gap persists.","supporting_citations":[{"cited_title":"2025 , url =","cited_arxiv_id":null,"evidence_quote":"Provides the Wan 2.2 diffusion-transformer backbone whose cross-attention is harvested as the interaction prior."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}