{"id":"11921923-a93a-4a0b-a57b-94d5fb9c5df9","arxiv_id":"2607.04653","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Role-aware joint training plus modality-decoupled denoising and loss annealing improves physical consistency of pretrained video diffusion models while preserving visual quality.","lead":"VPT fine-tunes video diffusion models with role-aware entity maps and modality-decoupled denoising so generated videos better obey physical interactions. It is a practical path to more plausible object motion, contact, and fluid dynamics without collapsing visual quality.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Role-map supervision may inject VLM/SAM labeling artifacts rather than true physical priors; the large SA jump is not cleanly isolated from that noise.","rationale":"The Reader correctly flags automatic role/flow labels as the weakest assumption and keeps CONDITIONAL with medium correctness risk. That is the right load-bearing concern: the headline gains, especially the SA jump attributed to the role map in Table 3(b), rest on treating imperfect VLM/SAM outputs as physical ground truth, while the paper only validates VAE reconstruction of those maps (Table 3c, Appendix B), not their physical correctness. No mathematical contradiction or circular derivation appears; the method is a coherent fine-tuning recipe with ablations against naive joint training and VideoJAM. Absent a label-quality or label-shuffle control, one cannot yet separate true physical-prior learning from fitting noisy semantic structure. That keeps the verdict CONDITIONAL rather than ACCEPT or REJECT; the concrete human-relabel + shuffle retrain would settle whether the concern lands. I agree with the Reader on the weakest assumption and do not move the verdict.","tokens_in":21073,"tokens_out":636,"duration_ms":6779,"concrete_test":"On a stratified 100-clip subset of WISA/VideoPhy training videos, have 2–3 human annotators re-label agent/controlled/passive/background and compute agreement with Qwen3-VL+SAM3 maps (IoU / role accuracy). Then retrain the Table 3(b) +Role variant with (i) human-corrected maps and (ii) deliberately shuffled role labels; if SA/PC under shuffled labels stays near 66.5/25.0, or human-corrected maps do not improve PC over automatic maps, the role-prior claim is not supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that role-aware joint training injects entity-level physical priors that improve VideoPhy SA/PC (47.7/21.2 → 66.5/25.0 on Wan2.1-1.3B). The load-bearing step is that Qwen3-VL role labels + SAM3 masks are accurate enough that supervising on them teaches agents/controlled/passive dynamics rather than correlating generation with VLM/SAM artifacts. Table 3(b) attributes most of the SA gain to the role map (+optical flow: 49.3/22.4; +role: 66.5/25.0), yet Limitations only notes the dependency and Table 3(c)/Fig. 5 only show VAE reconstruction fidelity of the maps, not label correctness or causal physical content. If role labels systematically mis-tag contact/agency (common for fluid/solid and multi-object scenes in VideoPhy), the SA lift can be semantic/layout alignment to noisy labels rather than physical consistency. Cross-step auto-guidance and modality decoupling do not resolve this: they amplify whatever was absorbed during fine-tuning on those labels.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes VPT, a lightweight fine-tuning framework that improves physical consistency of pretrained video diffusion models (Wan2.1-T2V 1.3B/14B). It augments RGB latents with optical flow and a four-class role-aware map (agent, controlled object, passive object, background), trains with modality-decoupled timesteps and cosine-annealed auxiliary losses so auxiliaries act as soft constraints, and applies cross-step auto-guidance at inference using an intermediate checkpoint. On VideoPhy the 1.3B model improves SA/PC from 47.7/21.2 to 66.5/25.0 (relative +39.4%/+17.9%), with further gains on VideoPhy-2 and competitive or improved VBench quality; ablations attribute most of the SA lift to the role map and show naive joint training collapses.","tokens_in":21414,"tokens_out":1274,"duration_ms":15594,"significance":"If the gains reflect genuine entity-level physical priors rather than label or layout artifacts, VPT is a practical and timely contribution: it targets a recognized failure mode of video diffusion (long-range dynamics and contact) with a modular fine-tuning recipe that preserves pretrained visual quality, is demonstrated at two model scales, and is supported by clear motivation experiments (capacity conflict, inference error accumulation) and component ablations. Strengths include external-benchmark evaluation (VideoPhy, VideoPhy-2, VBench), explicit comparison to VideoJAM and full fine-tuning, planned code/checkpoint release, and a limitations section that states dependence on RAFT/Qwen3-VL/SAM3. The work is of clear interest to the video-generation and world-model communities even if some claims need tighter causal support.","major_comments":[{"comment":"Table 3(b) attributes the dominant SA gain to the role-aware map (baseline 45.1/20.9 → +flow 49.3/22.4 → +role 66.5/25.0). The central physical-prior claim therefore depends on Qwen3-VL role labels and SAM3 masks being accurate enough to teach agent/controlled/passive dynamics rather than VLM/segmentation artifacts. Limitations only notes this dependency; Table 3(c) and Fig. 5 only measure VAE reconstruction fidelity of the maps, not label correctness or causal physical content. Please add (i) quantitative or human validation of role maps on a VideoPhy-relevant subset (especially solid–fluid and multi-object contact), and/or (ii) a control that randomizes or shuffles role labels while keeping masks/flow fixed, to show that the SA/PC lift requires correct role semantics rather than extra spatial structure.","section":null},{"comment":"§3.4–3.5 and Appendix A: modality-decoupled training is motivated as converting auxiliaries into soft constraints and mitigating recursive inference error, yet inference still constructs a joint latent with shared timesteps (tv=tf=tr) and recursively predicts flow/role channels (Eqs. 3–4; Inner-Guidance comparison). Table 3(a) shows cross-step auto-guidance improves PC (25.0→26.5) but slightly lowers SA (66.5→64.5). Please clarify what is actually sampled at inference (are auxiliary channels still denoised end-to-end?), report a pure text-only / auxiliary-dropped inference baseline under the same trained weights, and state whether the claimed reduction of recursive error is mainly from annealing during training or from auto-guidance.","section":null},{"comment":"Table 1 and §4.3: full fine-tuning of Wan2.1-1.3B can lower PC (21.2→20.9) while VPT raises it, but there is no matched compute/data ablation that applies the same WISA-80K data and LoRA budget without role/flow supervision under the same decoupled schedule. Without that control, part of the SA jump could be domain adaptation to WISA rather than role-aware physics. A “data-only / LoRA-only” row (or re-use of the Table 3(b) baseline with identical protocol) should be stated explicitly next to VPT.","section":null}],"minor_comments":[{"comment":"No error bars, confidence intervals, or multi-seed variance are reported (checklist item 7). Even a small multi-seed or multi-prompt subsample on VideoPhy would strengthen the large SA claim.","section":null},{"comment":"Fig. 1c–d and Fig. 4 are informative but axis labels and the meaning of “standard vs oracle” in 1d could be stated more explicitly in the caption for readers skimming.","section":null},{"comment":"Role scalar encoding {0, 85, 170, 255} and auxiliary dropout p are free design choices (§3.3); a short sensitivity note (or appendix) would help reproducibility.","section":null},{"comment":"Related work could more sharply contrast VPT with VideoREPA, PhyT2V, and WISA beyond a short list, given WISA-80K is the training set.","section":null},{"comment":"Typo/consistency: abstract and intro use “VPT” expanded as Video Physical-consistency Tuning; ensure hyphenation and capitalization are uniform throughout.","section":null}],"recommendation":"major_revision","confidential_remarks":"The skeptic concern about role-label noise is the main load-bearing risk; if the authors can add a shuffle/random-role control or modest human validation of maps, the paper becomes much stronger and closer to minor revision. Novelty relative to VideoJAM is real (roles + decoupled timesteps + annealing + auto-guidance) but incremental; fit for a solid CV/ML venue is good after the requested controls. No integrity or citation-pattern concerns."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean method paper on a real pain point: video diffusion still looks good frame-by-frame while breaking contact, fluid response, and agent–object dynamics. The useful takeaway is a lightweight fine-tune on Wan2.1 that lifts VideoPhy SA/PC a lot (47.7/21.2 → 66.5/25.0 on 1.3B; similar direction on 14B and VideoPhy-2) without the visual collapse you get from naive joint appearance–motion training.\n\nWhat is actually new is not “add optical flow.” VideoJAM already did joint motion. The distinctive pieces are (1) a four-role map (agent / controlled / passive / background) built with Qwen3-VL + SAM3, (2) independent noise levels per modality plus cosine annealing of the auxiliary loss so auxiliaries act as soft constraints, and (3) cross-step auto-guidance that uses an earlier checkpoint instead of recursive auxiliary CFG. Motivation plots on capacity conflict and inference error accumulation are honest and match the design. Ablations attribute most of the SA jump to the role map; modality decoupling is what keeps the pretrained visual prior from dying. VBench stays competitive. Citations and related-work framing are fair.\n\nSoft spots, in proportion: no error bars or multi-seed runs (expensive, but still a gap); only Wan2.1 backbones; free knobs (λ0, γ, intermediate step, role scalars) are reported but not deeply swept. The stress-test concern is real and is the main one: Table 3b shows the role map drives the big SA lift, yet the paper only proves VAE reconstruction fidelity of those maps, not that the VLM/SAM labels are physically correct. If labels systematically mis-tag agency or contact, you can get semantic/layout alignment to noisy tags rather than true physics. The authors flag the dependency in Limitations; they do not fully close it. That does not make the result circular—training data and VideoPhy evaluators are external—but it does mean the “physical prior” claim is only as good as the automatic labels.\n\nWho it is for: people already fine-tuning video diffusion for motion/physics and anyone comparing against VideoJAM-style joint training. Math is standard velocity MSE; no load-bearing formal claim. I would send it to referees. Worth engaging if you care about practical physical-consistency fine-tunes; treat the role-map causal story as provisional until label quality or human checks are stronger.","headline":"Solid fine-tuning recipe with real VideoPhy gains; the role-map is the main lever and also the main open risk, not a math failure.","tokens_in":22062,"tokens_out":602,"would_cite":true,"duration_ms":6201,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"VPT fine-tunes video diffusion models so agents, controlled objects, passive objects, and background keep physically coherent motion without collapsing visual quality.","keywords":["video diffusion","physical consistency","role-aware representation","modality-decoupled denoising","optical flow","cross-step auto-guidance","text-to-video","fine-tuning"],"falsifier":"Re-run the same fine-tuning protocol after deliberately corrupting or randomizing the role maps and optical flow; if SA/PC gains on VideoPhy disappear or reverse while visual quality stays high, the claimed physical priors are not the source of the improvement.","tokens_in":21942,"feed_emoji":"🎬","tokens_out":878,"duration_ms":7849,"temperature":0.7,"pith_summary":"Modern video generators look sharp frame by frame but still break physics: poured wine flies wrong, gloves miss balls, ripples look fake. Earlier fixes that jointly train on optical flow with the video itself often blur the pretrained look, treat every moving region the same, and accumulate prediction errors at sampling time. VPT is a lightweight fine-tuning recipe that tags entities by physical role, denoises the video and the auxiliary signals at independent noise levels so the auxiliaries act as soft constraints, anneals their loss weight so they do not overpower appearance, and uses an earlier training checkpoint to steer the final model toward stronger dynamics. On standard physical-commonsense benchmarks the method raises semantic adherence and physical scores while VBench quality stays competitive, showing that role structure plus decoupled supervision can inject motion priors without wrecking the original generator.","feed_headline":"Video models learn physics roles without losing visual quality","feed_subtitle":"Role maps plus decoupled denoising lift physical scores on VideoPhy while VBench quality holds","key_machinery":"Role-aware joint representation plus modality-decoupled denoising: optical flow and a scalar role map are VAE-encoded, concatenated with video latents, and each modality receives its own diffusion timestep; auxiliary losses are cosine-annealed so they become soft physical constraints rather than hard synchronized targets.","core_discovery":"VPT claims that a pretrained video diffusion model can be made substantially more physically consistent by (1) concatenating a four-role map (agent, controlled object, passive object, background) with optical flow, (2) training with independent noise schedules and decaying auxiliary loss so auxiliaries guide rather than dominate, and (3) applying cross-step auto-guidance at inference. On Wan2.1 backbones this lifts VideoPhy SA/PC from 47.7/21.2 to 66.5/25.0 (1.3B) and yields consistent gains on VideoPhy-2 while preserving or improving VBench quality.","pith_inferences":["If role maps prove brittle on long or multi-agent clips, the same decoupling machinery could be reused with denser 4-D geometry or force-field auxiliaries.","The capacity-conflict diagnosis suggests any multi-modal fine-tune of large video models may benefit from independent schedules rather than shared timesteps.","A natural next measurement is whether the learned role bias transfers to zero-shot prompts outside the WISA training distribution."],"forward_implications":["Lightweight post-training can raise physical-commonsense scores on material-interaction and action-centric benchmarks without full model re-pretraining.","Treating different entity roles separately yields larger gains than optical-flow-only joint training on the same backbone.","Independent noise levels and loss annealing reduce the visual collapse that naive joint appearance-motion training produces.","Cross-step auto-guidance from an intermediate checkpoint can strengthen dynamics more reliably than guiding on recursively predicted auxiliary latents."],"fun_headline_variants":["Role maps and decoupled noise raise video physical consistency","VPT lets video models learn entity physics roles without quality loss","Four-role signals plus soft auxiliaries fix long-range dynamics","Independent noise schedules keep visual priors while guiding physics","Role-aware fine-tuning lifts VideoPhy SA and PC scores"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The automatically extracted optical flow, vision-language role labels, and segmentation masks are accurate enough that training on them teaches real physics rather than label noise.","fun_headline_variants_meta":{"raw":{"variants":["Role maps and decoupled noise raise video physical consistency","VPT lets video models learn entity physics roles without quality loss","Four-role signals plus soft auxiliaries fix long-range dynamics","Independent noise schedules keep visual priors while guiding physics","Role-aware fine-tuning lifts VideoPhy SA and PC scores"]},"model":"grok-4.5","effort":"low","cost_usd":0.005348,"raw_usage":{"total_tokens":1506,"prompt_tokens":922,"num_sources_used":0,"completion_tokens":85,"cost_in_usd_ticks":53480000,"prompt_tokens_details":{"text_tokens":922,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":499,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":922,"tokens_out":85,"duration_ms":4200,"temperature":1.0,"reasoning_tokens":499,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T15:43:34.923256+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-run the same fine-tuning protocol after deliberately corrupting or randomizing the role maps and optical flow; if SA/PC gains on VideoPhy disappear or reverse while visual quality stays high, the claimed physical priors are not the source of the improvement.","supporting_citations":[],"review_version":1}