{"id":"76ee39a1-df39-4ff9-9ac1-10a156ed7d5b","arxiv_id":"2506.18655","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":8.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"RDPO builds preference pairs by reverse-sampling real video latents with a pre-trained generator, then fine-tunes with Flow-DPO, improving physics consistency metrics on two video models.","lead":"This paper introduces RDPO, a training method that uses real videos to create preference pairs automatically for video generation models, removing the need for human annotation. It then fine-tunes the generator with direct preference optimization to favor physically consistent motion.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 omits the text condition c that Eq. (3) optimizes; RDPO's preference signal may be reconstruction/SFT distillation, so the central transfer claim needs a prompt-conditional ablation.","rationale":"The paper's contribution is an annotation-free way to build preference pairs; if the pairs do not respect the text condition used at inference, the entire DPO objective is mismatched. The concern is not that RDPO is 'just SFT' in a pejorative sense—that could still be a useful method—but the manuscript explicitly frames Eq. (3) as maximizing pθ(x_preferred|c) relative to pθ(x_model|c) (Section 3.2.2), and Algorithm 1 never supplies c. The selection of x_preferred by L2 distance from x_model_t also makes no use of c, and Figure 6's human validation evaluates unconditional physical fidelity, not text-conditioned generation. Table 1's comparison to SFT is not decisive: SFT on original real videos is a different baseline from training on the partial-denoised latents that RDPO actually consumes. Thus the reported PhysicsIQ gains are compatible with the hypothesis that the model is simply being pulled toward the real-video manifold, with the DPO negative term contributing little. These are testable by code inspection and a three-way ablation. I do not think this warrants rejection—the idea is plausible and the human study provides partial support—but it does keep the verdict conditional, exactly as the reader concluded.","tokens_in":10993,"tokens_out":7421,"duration_ms":91746,"concrete_test":"Check the released RDPO code and trace the prompt argument through Algorithm 1's reverse-sampling calls at lines 3 and 11. Then run a three-way LTX-2B iter1 ablation: (a) RDPO as implemented; (b) same pipeline but with x_preferred generated using the real video's original caption as c and the Flow-DPO term replaced by pure SFT on x_preferred (drop the x_model term); (c) RDPO with x_preferred generated under an empty/global prompt. Report PhysicsIQ and Total VBench with 95% CIs over at least 5 seeds. If (b) matches (a), the preference term is not the source of the gains; if (c) matches (a), the prompt path is irrelevant; if neither, the concern resolves in the paper's favor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"For the central claim—that annotation-free pairs improve text-conditioned physics consistency via preference optimization—two things must hold: x_preferred must actually be drawn from (or at least be a valid positive for) the conditional policy πθ(·|c), and the DPO signal must add something beyond supervised fitting of real-video latents. Algorithm 1 does not establish either. Lines 3 and 11 call the denoiser πθ with only a latent and a timestep; no prompt argument appears, and the selection criterion d=||x_t−x_model_t|| (line 7) is purely a latent-space distance. If reverse sampling is not conditioned on c, then log πθ(x_preferred|c) in Eq. (3) is the likelihood of a sample that was never generated from that condition, so the loss is either mis-specified or silently relies on an undocumented prompt path. The supporting human study (Appendix A.1, Figure 6) shows that partially denoised real-video latents look more physically consistent than full-noise samples, but it does not test whether that preference survives when the positive and negative videos are compared under the same text prompt used in VBench/PhysicsIQ evaluation. Finally, Table 1's isolation experiment does not rule out the distillation reading: RDPO(w/o sft) is 24.71 PhysicsIQ versus 24.63 for SFT on real videos, a 0.08-point gap with no error bars, and neither baseline trains on the partial-denoised positives that RDPO actually uses. The current experiments therefore cannot distinguish 'preference learning from real-video priors' from 'mild supervised distillation of real videos with an extra negative log-ratio term.'","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RDPO, an annotation-free post-training method for text-to-video diffusion models that aims to improve physical consistency. Preference pairs are constructed automatically by (i) generating a negative video from full Gaussian noise and (ii) generating a positive video by partially noising a real video and reverse-sampling from an intermediate timestep, with rejection sampling selecting the latent closest to the model's own sample. The pairs are used with a DPO-style loss in Eq. (3), interleaved with SFT, in a progressive curriculum that reduces the reverse-sampling step over iterations. Experiments on LTX-2B report improved PhysicsIQ and VBench scores, and experiments on Shopee-MUG-V-10B report favorable human evaluation and automatic metrics.","tokens_in":11330,"tokens_out":3901,"duration_ms":46085,"significance":"If the claims hold, RDPO is a practically valuable way to use real videos as preference supervision without human annotations or reward models, and the progressive curriculum is a sensible coarse-to-fine strategy. The paper has clear strengths: it provides an explicit algorithm, releases code, evaluates on two model scales, validates pair quality with a human study, and compares against human-annotated DPO. The main risk is whether the signal is genuinely conditional preference learning from real-video priors rather than a form of supervised reconstruction distillation; the current experiments do not fully separate these explanations.","major_comments":[{"comment":"The preference-pair construction pipeline never passes the text condition c through the sampling calls: lines 2, 3, 6, and 11 invoke f and πθ only with latents and timesteps, while the DPO objective in Eq. (3) evaluates log πθ(x_preferred|c). As written, x_preferred is not established to be a sample from the conditional policy πθ(·|c), so it is unclear what likelihood ratio the loss optimizes. Please specify how prompts condition reverse sampling, and include an ablation comparing unconditional versus prompt-conditional reverse sampling to verify transfer to text-conditioned generation.","section":"Algorithm 1, Eq. (3)"},{"comment":"The 0.08-point PhysicsIQ gap between RDPO (w/o sft) and SFT (24.71 versus 24.63) is too small, and no error bars or significance tests are reported, to support the claim that preference learning from real-video latents adds something beyond supervised fitting. The comparison is also confounded: SFT is trained only on clean real videos, while RDPO's positives are partially denoised latents. Add an SFT baseline trained on the same x_preferred positives with the same data budget, and report variance across seeds or evaluation sets.","section":"Section 4.1.2, Table 1"},{"comment":"The claim that annotation-free RDPO outperforms human-annotated DPO is not controlled for data quantity: RDPO uses 8,000 pairs versus 1,000 human-labeled pairs, and no confidence intervals are given. Please add a matched-size human DPO training run (or a learning-curve comparison) and report significance tests, since the observed differences on PhysicsIQ (25.21 versus 24.70) and Total Score (85.63 versus 85.44) are small.","section":"Section 4.1.3, Table 2"},{"comment":"The progressive schedule (42, 40, and a mixture of both) is chosen by inspecting Total Score on the same benchmarks used for final reporting, so the reported improvements are selected, not predicted, results. Please use a held-out validation set for schedule selection, or report all evaluated configurations and mark the selected one, to avoid selection bias in the headline numbers.","section":"Section 4.1.4, Figure 3 and Table 3"}],"minor_comments":[{"comment":"The heading 'A Dissusion' contains a typo; it should read 'A Discussion'.","section":"Appendix A"},{"comment":"The human validation study does not report the number of annotators, the number of pairs evaluated, or error bars on the accuracy curve; please add these details.","section":"Section A.1, Figure 6"},{"comment":"Algorithm 2 unconditionally interleaves DPO and SFT every other step, but Table 1 reports a configuration called 'RDPO (w/o sft)'; please clarify how that configuration was trained.","section":"Algorithm 2 and Table 1"},{"comment":"The comparison in Table 4 only contrasts Shopee-MUG-V-RDPO against the base Shopee-MUG-V; please clarify whether the column heading 'other open-source model' is intended to indicate additional baselines and, if so, why they are not reported.","section":"Section 4.2.2, Table 4"},{"comment":"References [42] and [43] are the same paper; please remove the duplicate. Also, please provide a URL or release status for the WISA dataset if it is intended to be public.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The evaluation on Shopee-MUG-V-10B is on a proprietary model from the same lab, so it is less informative for external validation; the LTX-2B experiments should carry the main weight. The reader's stress-test concern about prompt conditioning in Algorithm 1 is genuine and needs to be addressed before the central transfer claim can be accepted. The small absolute gains and lack of error bars are also concerning, but they are fixable with additional experiments and reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"RDPO has a genuinely clever core idea: partially noise a real video, reverse-sample it with the frozen generator, and label the result 'preferred' over the model's own full-noise sample. That gives you preference pairs for DPO at zero annotation cost. The combination—reverse-sampling real latents plus selection for closeness to the model manifold—is new as far as I know, and it's an appealing way to inject physical priors.\n\nThe paper does a few things well. The human validation in Appendix A.1 confirms that the reverse-sampled clips are indeed judged more physically consistent than the model's own samples, which is the load-bearing assumption. The authors also test on two model sizes (LTX-2B open, Shopee-MUG-V-10B proprietary), and they are upfront about limitations. The self-citation to IPO is not a problem by itself; the overlapping authorship is fine when the cited result is real.\n\nBut the evidence doesn't yet establish the central claim that this is preference learning rather than distillation. Algorithm 1 never passes the text prompt to the denoiser—πθ is called with only a latent and a timestep—while Eq. (3) optimizes log πθ(x|c). If the preferred sample is generated without conditioning on c, the DPO loss is mis-specified. Maybe the implementation does condition on the prompt, but the paper needs to say so and ablate it. This is the first load-bearing soft spot.\n\nSecond, the isolation experiment in Table 1 is too weak to rule out the SFT/distillation reading. RDPO (w/o sft) is 24.71 PhysicsIQ versus 24.63 for SFT on real videos—a 0.08 gap with no error bars. Neither baseline trains on the partial-denoised positives that RDPO actually uses, so you cannot distinguish 'preference signal' from 'mild supervised regression to real videos.' Add a baseline that SFTs on the same x_preferred positives.\n\nThe other issues are standard but still need fixing: no significance tests or confidence intervals; the human-DPO comparison uses 1,000 labeled pairs versus 8,000 RDPO pairs, so it's an unfair data-scaling comparison; and the reverse-step schedule (42, 40, mixture) looks tuned on the very benchmarks reported. The 10B model results rely on a human study with no detail on rater count or agreement.\n\nBottom line: this is a promising direction and the core idea deserves a serious referee. But as written, the claims overreach. I'd send it to review and then demand major revision: clarify the prompt path in Algorithm 1, add a prompt-conditional ablation and an SFT-on-positives baseline, report error bars, and equalize data scales. If those checks hold up, RDPO could be a useful tool for physics-aware video alignment.","headline":"Clever annotation-free preference construction, but the paper doesn't yet prove it's preference learning rather than real-video distillation.","tokens_in":11889,"tokens_out":4379,"would_cite":true,"duration_ms":45190,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RDPO shows that preference optimization for video physics can run on real-video priors alone, without human annotation.","keywords":["video generation","physics consistency","preference optimization","DPO","annotation-free","diffusion models","rectified flow","real data prior"],"falsifier":"Take the trained RDPO model and feed it preference pairs where x_preferred is replaced by the original real video (or a fully denoised copy of it). If the PhysicsIQ gain disappears or is unchanged compared to plain SFT on real videos, the preference signal carries no information beyond the real-video prior itself.","tokens_in":10805,"feed_emoji":"🎬","tokens_out":4038,"duration_ms":39221,"temperature":0.7,"pith_summary":"This paper tries to establish that a video generation model can be taught real-world physics without any human annotation, using the model's own relationship to real videos. The claim is that by partially noising real footage and then reverse-sampling it with the pretrained generator, one can automatically produce a 'preferred' video that is statistically more physically correct than a video generated from pure noise. Training the generator with a direct preference optimization objective on these self-built pairs improves physical realism and overall video quality. If true, it removes the main bottleneck of preference-based post-training for video: costly human labels or brittle reward models.","feed_headline":"Real-video priors beat human-annotated DPO on video physics","feed_subtitle":"Automatically pairing partially denoised real videos with generated ones lifts PhysicsIQ from 24.40 to 25.66 on LTX-2B.","key_machinery":"The load-bearing mechanism is reverse sampling from a real video's noised latent (an idea borrowed from image-to-image editing), steered by rejection sampling: among K noised versions of the real latent at timestep t, the one closest in L2 distance to the model's own sample at that timestep is chosen for denoising. This keeps the 'preferred' video aligned with the model's manifold while injecting real-world dynamics. The Flow-DPO objective (Equation 3) then raises the likelihood of x_preferred relative to x_model, and a progressive curriculum—starting from heavy noise and decreasing reverse steps across iterations—pulls the model toward the real-data distribution.","core_discovery":"On the paper's own terms, the central discovery is that preference pairs for physics consistency can be manufactured from real videos at no annotation cost. The method takes a real video, adds noise to a chosen intermediate timestep, and reverse-samples that noisy latent to get x_preferred, while a normal generation from full noise yields x_model. Because the reverse-sampled output retains dynamic structure from the real footage, it is usually closer to physical plausibility; the pair (x_preferred, x_model) then serves as a synthetic DPO example. Applied with a coarse-to-fine schedule of reverse steps and combined with SFT, this raises LTX-2B's PhysicsIQ from 24.40 to 25.66 over three iterations and matches or exceeds human-annotated DPO on PhysicsIQ and Total Score.","pith_inferences":["A likely implicit limit: because Algorithm 1 does not condition the reverse sampling on the text prompt, x_preferred may not correspond to the same semantic content as x_model's prompt; if the pair does not share the prompt c, the DPO update could be teaching prompt-agnostic imitation rather than physics per se.","One could test whether the gains scale with diversity of real videos: if the physics prior is purely distributional, a wider set of real scenes should yield larger PhysicsIQ improvements; if not, the method may be memorizing specific dynamics.","The rejection-sampling selection in latent space resembles a mini-max choice; an interesting extension would be to make the selection differentiable or to use a learned similarity metric rather than L2 in latent space, potentially improving the stability-precedence tradeoff."],"forward_implications":["If correct, any text-to-video diffusion model can be post-trained for physics consistency without building reward models or collecting human preferences, using only a corpus of real videos.","The progressive reverse-step curriculum suggests a general recipe: start with high noise to teach global dynamics, then decrease noise for fine-grained physical detail, interleaving earlier phases to avoid catastrophic forgetting.","RDPO's auto-built pairs are complementary to human-labeled DPO: mixing them gives the best scores, implying the two supervision sources carry partially independent signal.","The approach may transfer to other attributes beyond physics—whatever distributional structure real videos carry—provided the dataset is curated for that attribute."],"supporting_citations":[{"why":"Provides the rectified-flow formulation and the velocity-field ODE that RDPO's noise-injection and reverse sampling operate on, and the Flow-DPO loss is defined over these trajectories.","marker":"[23]"},{"why":"Supplies the direct preference optimization objective for diffusion models, which RDPO adapts as Flow-DPO for video.","marker":"[32]"},{"why":"The LTX-2B baseline video generator that RDPO is applied to and evaluated on.","marker":"[10]"},{"why":"The PhysicsIQ benchmark that provides the physical-realism metric central to the paper's improvement claims.","marker":"[25]"},{"why":"The VBench++ benchmark used for overall video-quality evaluation and the Total Score metric.","marker":"[15]"},{"why":"The image-editing idea of denoising a partially noised real input, which RDPO extends to video for building preferred samples.","marker":"[24]"}],"fun_headline_variants":["No labels needed: Real videos teach AI video physics","RDPO: Real videos become free preference pairs","Physics from reality: AI learns without human ratings","Reverse-sampling real clips boosts video physics scores","Annotation-free DPO lifts PhysicsIQ by 1.26 points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on the assumption that a video produced by partially denoising a real video's latent is consistently more physically correct than a video generated from full noise, and that this preference survives text-conditioned generation; if the 'preferred' video is essentially just a reconstruction of the real footage, the DPO signal collapses into supervised fine-tuning.","fun_headline_variants_meta":{"raw":{"variants":["No labels needed: Real videos teach AI video physics","RDPO: Real videos become free preference pairs","Physics from reality: AI learns without human ratings","Reverse-sampling real clips boosts video physics scores","Annotation-free DPO lifts PhysicsIQ by 1.26 points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000573,"raw_usage":{"total_tokens":2682,"prompt_tokens":895,"completion_tokens":1787,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":1720}},"tokens_in":511,"tokens_out":1787,"duration_ms":15065,"temperature":1.0,"reasoning_tokens":1720,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:15:34.991881+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained RDPO model and feed it preference pairs where x_preferred is replaced by the original real video (or a fully denoised copy of it). If the PhysicsIQ gain disappears or is unchanged compared to plain SFT on real videos, the preference signal carries no information beyond the real-video prior itself.","supporting_citations":[{"cited_title":"Diffusion model alignment using direct preference optimization","cited_arxiv_id":null,"evidence_quote":"Supplies the direct preference optimization objective for diffusion models, which RDPO adapts as Flow-DPO for video."}],"review_version":1}