{"id":"0ede92ed-551a-4b31-8fe8-043a830ce5d6","arxiv_id":"2602.01624","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"PISCES post-trains text-to-video models using dual optimal-transport-aligned rewards (global quality plus token-level semantic) and outperforms annotation-based and annotation-free baselines on VBench and human evaluation.","lead":"PISCES is a fine-tuning method that uses optimal transport to shift text embeddings into the video embedding space before computing rewards, letting text-to-video models be improved without human preference labels. In tests, it beats both annotation-based and annotation-free post-training baselines on VBench and in human preference ratings for short and long videos.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"R_OT-quality may be an unconditional realism reward: no control separates text-conditioned OT alignment from simple projection onto the real-video manifold.","rationale":"The paper's central claim is that annotation-free OT-aligned rewards let PISCES outperform both annotation-based and annotation-free baselines. For that claim to hold in the sense argued, the distributional OT map must preserve per-prompt text-video semantic identity; otherwise R_OT-quality is a generic realism signal and the novelty is substantially weakened. The reader identified this assumption as weakest. I agree it is a real soft spot, but I would sharpen it: the paper's Table 4 does include Mutual KNN, which is a per-pair retrieval-style statistic, so it is not true that only Spearman correlation is checked. However, a single aggregate Mutual KNN value (0.26) does not establish that the OT target used in Eq. 2 is semantically faithful for each prompt, nor does it rule out the simpler explanation that projecting text embeddings onto the real-video manifold—regardless of where they land—is what improves quality scores. The proposed control with a prompt-independent real-video [CLS] target would settle this by isolating text-conditioning from manifold projection. Other concerns (missing error bars, no released code, possible VTM reward hacking) are real but secondary or partially mitigated by the CD-loss ablation and human study. Since the reader's verdict is already CONDITIONAL and this concern supports that verdict without requiring a stronger rejection, the verdict should remain unchanged.","tokens_in":25505,"tokens_out":17401,"duration_ms":175936,"concrete_test":"Run the identical post-training pipeline (same base model, data, LoRA, CD loss, semantic reward) but replace R_OT-quality in Eq. 2 with cosine similarity to a fixed, prompt-independent real-video [CLS] target, e.g., the mean real-video [CLS] over the training set (or a randomly sampled real-video [CLS] per step). Compare VBench Quality/Total and human preferences against full PISCES across, say, 3 seeds. If the control matches PISCES within seed-level noise, R_OT-quality is not text-conditioned and the OT map's per-prompt correspondence is not carrying the quality gains; if PISCES is clearly better, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The quality reward in Eq. 2 rests on the proxy T*(y[CLS]) ≈ x_real[CLS] for the same prompt. The OT map is trained from the marginals of text and video embeddings (Eq. 1, Algorithm 2), with no paired constraint tying a text to its matching real video. OT is not unique, and the reported checks do not close the gap: Spearman 0.90 only says the text-side geometry is rank-preserved; the Mutual KNN in Table 4 is an aggregate retrieval statistic, not a demonstration that the specific target used for reward computation is the paired (or a semantically equivalent) real-video embedding for each prompt. If T* sends y to a video-manifold point that is independent of the prompt's meaning, R_OT-quality is an unconditional realism reward: it pulls generated videos onto the real-video manifold but does not condition on text. Then the improvement over raw-cosine baselines could be explained by the target being moved onto the video manifold, not by OT's semantic alignment. The paper contains no control that replaces T*(y[CLS]) with a prompt-independent real-video [CLS] (e.g., mean or random real-video embedding). This matters because the central claim attributes the gains to OT-aligned rewards, and the quality-reward ablation (Table 3) is the main evidence for that attribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PISCES proposes an annotation-free post-training method for text-to-video diffusion models, built around a Dual Optimal Transport-aligned Rewards module. A distributional OT map is trained from text and video embedding marginals and used in a cosine-similarity Quality Reward (Eq. 2), while a token-level partial-OT plan is injected into the cross-attention of an InternVideo2 VTM classifier to produce a Semantic Reward (Eq. 4). The paper reports VBench and human-preference gains over annotation-based and annotation-free baselines on VideoCrafter2 and HunyuanVideo, with direct-backpropagation and GRPO optimization variants, plus extensive appendices on ablations, hyperparameters, OOD behavior, and training cost.","tokens_in":25850,"tokens_out":7612,"duration_ms":79305,"significance":"If the central attribution is correct, this is a meaningful contribution: it suggests that carefully aligned embeddings can make annotation-free reward supervision competitive with or superior to human-preference-based post-training, which would improve scalability. The manuscript is unusually thorough in its appendices: Algorithms 1-3 are given in detail, the method is tested under two optimization paradigms, with two video backbones, two embedding encoders, a ViCLIP-based generalization check, an OOD prompt study, inter-rater reliability, and a discussion of reward hacking. These are strengths. However, the headline claim rests on a proxy in Eq. (2) that is not directly validated, on point estimates without uncertainty quantification, and on hyperparameters selected with the same VTM metric used as reward evidence. These load-bearing issues need to be resolved before the contribution is established.","major_comments":[{"comment":"The quality reward rests on the proxy T*(y[CLS]) ≈ x_real[CLS] for the same prompt. The OT map is trained on marginal distributions of text and video embeddings (Eq. (1), Algorithm 2) with no paired constraint tying a given prompt to its matching real video. OT is non-unique, and the checks in Table 4 and Fig. 6 (Mutual KNN, Spearman, t-SNE) are aggregate geometry measures; they do not establish that the specific target used in Eq. (2) is the paired or semantically equivalent real-video embedding. If T* only projects text onto the video manifold while ignoring prompt semantics, R_OT-quality is an unconditional realism reward, and the quality gains in Table 3 would be explained by target manifold placement rather than OT alignment. Add a control replacing T*(y[CLS]) with a prompt-independent real-video embedding (mean or random) and report a retrieval-based per-prompt correspondence metri","section":"Section 3.1, Eq. (2)"},{"comment":"The hyperparameters m=0.9, γ=η=0.2 are selected by maximizing Video-Text Matching (VTM) accuracy on WebVid10M (Appendix J), and the same VTM accuracy is then reported as evidence that partial OT and spatio-temporal constraints improve alignment (Table 7). Because the semantic reward is defined as the VTM classifier logit, this is hyperparameter selection on the evaluation metric, not independent validation. Report the chosen configurations' VBench Quality/Semantic scores on held-out prompts, or use an alignment metric separate from the reward design, to support generalization of these constraints.","section":"Section 4.3, Appendices F and J"},{"comment":"The headline claim that PISCES 'significantly outperforms' annotation-based methods is based on point estimates without error bars, repeated seeds, or significance tests. Some margins are small (e.g., VideoCrafter2 Semantic: 77.54 vs VideoDPO 77.38; Total: 82.75 vs 81.93 in Table 1). The human-preference percentages in Fig. 3 likewise lack confidence intervals, although inter-rater reliability is reported in Appendix L. Please report variance over at least three seeds or bootstrap confidence intervals, and appropriate significance tests for the main comparisons.","section":"Section 4.2, Tables 1-2 and Fig. 3"}],"minor_comments":[{"comment":"The mapping from the 'effective transported fraction m' to the unbalanced Sinkhorn relaxation parameters τ_a, τ_b is stated but not derived. A short explanation of why τ(ρ)=ρ/(ρ+ε) implements the desired mass fraction would help reproducibility.","section":"Section 3.2 / Appendix C"},{"comment":"The Contrastive row leaves Spearman correlation empty. If not measured, say so explicitly; if it is undefined, explain why.","section":"Table 4"},{"comment":"Figure 8 labels the x-axis 'Configuration Index' but the text does not provide a mapping from index values to (γ, η) pairs. Including the actual grid values on the axis would make the sensitivity result directly interpretable.","section":"Appendix J"},{"comment":"The term 'annotation-free' is used to mean 'no human preference annotations,' but the method still relies on paired text-video data (WebVid10M, VidGen-1M) and a pre-trained VTM classifier. Clarify this terminology early to avoid overclaiming.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is promising and the appendix is unusually thorough, but the central attribution of the quality gains to OT alignment needs a decisive control experiment, and the statistical support for the 'outperforms annotation-based methods' claim is currently thin. If the proposed control shows that a prompt-independent target performs comparably to T*(y[CLS]), the contribution would reduce to a generic realism regularizer despite the OT framing. I recommend major revision rather than rejection because the issues are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PISCES is worth a look. It's a well-executed annotation-free reward pipeline for T2V post-training: train a Neural OT map to push text embeddings into the real-video embedding space, then use a cosine quality reward; add a token-level partial-OT plan with spatio-temporal costs fused into InternVideo2's VTM to get a semantic reward. The combination is new, and the paper does a lot of things right: ablations for each reward, two base models, direct backprop and GRPO, a human study with Fleiss Kappa, an alternative ViCLIP evaluator, OOD prompts, hyperparameter sweeps, and a frank discussion of a failure case where partial OT drops the 'glasses' token. The empirical story is coherent and the gains are consistent, even if some margins are modest.\n\nThe main soft spot is the quality reward's text-conditioning. Eq. (2) relies on T*(y[CLS]) ≈ x_real[CLS] for the same prompt, but the OT map is trained only on marginals (Eq. 1, Algorithm 2) with no paired supervision. OT is not unique, and the reported checks (Spearman ~0.90, Mutual KNN) are aggregate; they don't show that a specific prompt's OT target is semantically the corresponding real video. So R_OT-quality could be an unconditional realism score—pulling generations onto the video manifold without conditioning on text. The paper has no control replacing T*(y) with a prompt-independent real-video embedding. This doesn't sink the paper: the semantic reward does carry the text signal, and the ablations show OT helps. But the attribution of quality gains specifically to 'OT-aligned' rewards is weaker than advertised.\n\nOther, smaller issues: no error bars or significance tests on VBench (standard for the field, but some margins are small), hyperparameters like m=0.9 and the spatio-temporal weights are tuned on WebVid10M VTM accuracy—the same task the semantic reward optimizes—and no code or data are released, which slows independent checking.\n\nOverall, this is a solid, honest paper that deserves a serious referee. The quality-reward concern is addressable with a simple control, and if the authors add that, the claim becomes much stronger. I'd bring it to reading group and cite it.\n\nRecommend: send to peer review.","headline":"Well-executed annotation-free T2V post-training with a genuinely new dual-OT reward module; the quality reward's text-conditioning is asserted rather than proven, but the paper is honest, thorough, and deserves a serious referee.","tokens_in":26338,"tokens_out":2752,"would_cite":true,"duration_ms":27154,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Optimal transport can realign text and video embeddings so that annotation-free rewards for text-to-video post-training outperform human-preference-based methods.","keywords":["text-to-video generation","reward-based post-training","optimal transport","embedding alignment","semantic grounding","video-text matching","annotation-free alignment","quality reward"],"falsifier":"For prompts paired with both matching and non-matching videos, check whether the OT-transformed prompt embedding ranks the matching video's summary embedding closer than the non-matching one. If it does not, the quality reward is not text-conditioned, and gains attributed to the OT alignment would need to be explained by the semantic reward or by the consistency-loss regularizer.","tokens_in":25385,"feed_emoji":"🎬","tokens_out":8898,"duration_ms":89496,"temperature":0.7,"pith_summary":"This paper tries to prove that reward-based post-training for text-to-video generation can work without any human preference annotations, provided the reward is evaluated in an embedding space realigned with optimal transport. It proposes PISCES, with two rewards: a distributional quality reward that transports text embeddings onto the real-video manifold before taking cosine similarity with generated videos, and a token-level semantic reward that uses a partial optimal-transport plan with spatio-temporal costs to sharpen cross-attention in a video-text matching classifier. The paper reports that this dual scheme outperforms both annotation-based and annotation-free post-training methods on short and long video generators, on automatic metrics and in a human preference study. If right, this removes the main scalability bottleneck of reward-based video post-training: collecting large preference datasets.","feed_headline":"One OT step helps no-label video rewards beat preference-trained ones","feed_subtitle":"Transporting text prompts into real-video space improves quality and prompt fidelity without annotations.","key_machinery":"The central object is the dual optimal-transport alignment: a distributional transport map between text and video embedding distributions, trained with squared-Euclidean cost through a two-network minimax objective; and a discrete partial optimal-transport plan over text tokens versus video patch tokens, with a cost that penalizes semantic dissimilarity, temporal distance, and spatial distance, computed with an entropic solver and fused into cross-attention in log space. The distributional map carries the quality reward by making a transported text embedding comparable to a generated video embedding on the real-video manifold; the token-level plan carries the semantic reward by selecting whi","core_discovery":"The central claim is that the mismatch between pre-trained text embeddings and real-video embeddings, rather than the scarcity of paired preference data, is the bottleneck for annotation-free text-to-video post-training. PISCES corrects this mismatch in two complementary ways. At the distributional level, a neural optimal-transport map is trained to push text embeddings onto the real-video embedding distribution while preserving internal structure; the quality reward is then cosine similarity between the transported text summary embedding and the generated video's summary embedding, making the text embedding a proxy for a real video of that prompt. At the token level, a partial entropic opti","pith_inferences":["Not in the paper: if the transport map preserves only the aggregate distribution rather than each prompt's correspondence to its own real video, the quality reward could degenerate into a generic realism score; a per-item retrieval test would settle whether the OT alignment is doing text-conditioned work.","An editorial extension: the dual alignment recipe could transfer to other cross-modal post-training settings—image-text, audio-video, or embodied language—wherever frozen embeddings occupy misaligned marginal spaces.","The paper's Appendix K concedes that fine-grained token grounding is bounded by the frozen video-text encoder's spatial precision; reading forward, switching to a stronger base encoder should transfer directly into better grounding under PISCES.","A further implication: the log-space fusion of the transport plan with cross-attention is a structural prior that could be reused at inference time, for example to visualize which prompt words ground which video regions."],"forward_implications":["If correct, annotation-free post-training can match or surpass human-preference-based post-training, removing the main scaling bottleneck for video alignment.","Quality and semantic supervision separate cleanly: the quality reward drives global realism and temporal coherence, the semantic reward drives object and action grounding, and combining them gives the best of both.","Because the same reward module works under direct backpropagation and reinforcement-learning fine-tuning, existing post-training stacks can adopt it without changing the optimizer.","The paper's cost analysis implies the optimal-transport map is a small, one-time addition relative to model pre-training, and the token-level transport plan doubles as an inspectable alignment of words to video regions."],"fun_headline_variants":["OT aligns text-video embeddings, enabling annotation-free video post-training","Dual OT-aligned rewards beat preference-trained baselines in T2V","PISCES: Optimal transport fixes embedding mismatch for better video rewards","No labels needed: OT-transported text embeddings boost video quality","Bridging text and video spaces with OT outperforms human-preference training"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The quality reward assumes the transport map sends a given prompt's embedding close to a real video of that same prompt, but the map is trained only on the overall distributions of prompts and videos separately, with no pairing that ties a specific prompt to the video that matches it.","fun_headline_variants_meta":{"raw":{"variants":["OT aligns text-video embeddings, enabling annotation-free video post-training","Dual OT-aligned rewards beat preference-trained baselines in T2V","PISCES: Optimal transport fixes embedding mismatch for better video rewards","No labels needed: OT-transported text embeddings boost video quality","Bridging text and video spaces with OT outperforms human-preference training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1330,"prompt_tokens":811,"completion_tokens":519,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":425}},"tokens_in":555,"tokens_out":519,"duration_ms":6341,"temperature":1.0,"reasoning_tokens":425,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T05:35:41.622535+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For prompts paired with both matching and non-matching videos, check whether the OT-transformed prompt embedding ranks the matching video's summary embedding closer than the non-matching one. If it does not, the quality reward is not text-conditioned, and gains attributed to the OT alignment would need to be explained by the semantic reward or by the consistency-loss regularizer.","supporting_citations":[],"review_version":1}