{"id":"75b5065d-f13c-4a30-a320-051091fca023","arxiv_id":"2606.02564","paper_version":3,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"VLMs formulate differentiable rewards from task-specific rules to enable test-time online LoRA optimization of VGMs, delivering 16.7-point gains on symbolic and general video reasoning benchmarks over VLM-as-solver and Best-of-N baselines.","lead":"The paper proposes shifting VLMs from solvers to teachers that extract task rules to create differentiable rewards, which then guide test-time LoRA optimization of a VGM for video reasoning tasks. A smart generalist might read it to see how perception strengths in one model can adaptively steer generation in another at inference time without full textual plans.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Mechanism for converting VLM outputs into end-to-end differentiable rewards is unspecified","rationale":"The reader's weakest assumption directly identifies the same hinge (VLM perception → effective differentiable rewards). The concern is technical rather than perceptual accuracy alone: without the differentiability bridge, the test-time optimization cannot function as stated regardless of VLM quality.","tokens_in":1878,"tokens_out":302,"duration_ms":23277,"concrete_test":"In the methods section on reward formulation, extract the exact expression for the reward r(video). Replace any VLM call with a fixed non-differentiable surrogate (e.g., random score or precomputed lookup) and rerun the LoRA optimization on one benchmark split; if the reported gain falls below 5 points, the differentiability claim is unsupported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires VLMs to extract rules and produce differentiable rewards that guide gradient-based test-time LoRA optimization on the VGM. Standard VLMs yield discrete or non-differentiable scores; without an explicit construction (e.g., embedding similarity, soft proxy, or auxiliary head) that makes the reward a differentiable function of generated video latents, the optimization loop cannot propagate gradients from the VLM teacher. The abstract asserts this works and yields 16.7-point gains, but the missing construction is the load-bearing step that must hold for the teacher paradigm to be distinct from non-differentiable search.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that VLMs can serve as teachers rather than solvers for video reasoning by extracting task-specific rules to create differentiable rewards, which then guide test-time online optimization of a lightweight LoRA module on a Video Generation Model (VGM) Reasoner. This yields a reported 16.7-point average gain on symbolic (VBVR-Bench) and general-purpose (RULER-Bench) benchmarks, substantially outperforming VLM-as-Solver (+0.4) and Best-of-N scaling (+2.2) at comparable test-time cost.","tokens_in":1986,"tokens_out":521,"duration_ms":19330,"significance":"If the central mechanism holds, the work would represent a meaningful paradigm shift in video reasoning by repurposing VLMs' perceptual strengths for adaptive, gradient-based test-time adaptation rather than static textual guidance. The empirical gains, if reproducible with proper controls, would demonstrate a practical way to extend VGM reasoning boundaries without retraining.","major_comments":[{"comment":"The load-bearing step—conversion of VLM outputs into end-to-end differentiable rewards that permit gradient flow to the VGM's LoRA parameters—is not specified. Standard VLMs produce discrete or non-differentiable judgments; without an explicit construction (embedding similarity, soft proxy, auxiliary head, or equivalent) that makes the reward a differentiable function of generated video latents, the claimed test-time optimization loop cannot operate as described. This must be detailed with equations or pseudocode in the method section for the teacher paradigm to be distinct from non-differentiable search.","section":"Method / Reward Formulation"},{"comment":"§4 (Experiments): The abstract reports a 16.7-point average gain and specific outperformance margins, yet provides no details on experimental controls, error bars, exact reward formulations, data splits, or statistical significance. These omissions prevent verification that the gains support the central claim rather than arising from uncontrolled factors.","section":"Experiments"}],"minor_comments":[{"comment":"Clarify the precise VGM architecture, LoRA rank, optimization steps, and learning rate schedule used at test time, as these directly affect the claimed comparable test-time cost.","section":"Implementation Details"},{"comment":"Add missing references to prior test-time adaptation or reward-modeling work in video generation to better situate the contribution.","section":"Related Work"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback highlighting the need for greater clarity on the reward formulation and experimental details. We address each major comment below and will revise the manuscript accordingly.","responses":[{"response":"We agree that the explicit construction for differentiability is essential and currently underspecified in the manuscript. The high-level description of VLM-derived rewards is present, but the precise mechanism (e.g., embedding cosine similarity or soft proxies) enabling gradient flow to LoRA parameters is not detailed with equations. In the revision we will add a dedicated subsection with equations and pseudocode in the Method section to make this construction explicit and distinguish it from non-differentiable search.","revision_made":"yes","referee_comment":"[Method / Reward Formulation] The load-bearing step—conversion of VLM outputs into end-to-end differentiable rewards that permit gradient flow to the VGM's LoRA parameters—is not specified. Standard VLMs produce discrete or non-differentiable judgments; without an explicit construction (embedding similarity, soft proxy, auxiliary head, or equivalent) that makes the reward a differentiable function of generated video latents, the claimed test-time optimization loop cannot operate as described. This must be detailed with equations or pseudocode in the method section for the teacher paradigm to be distinct from non-differentiable search."},{"response":"We acknowledge that the current manuscript lacks sufficient experimental controls and reporting details. In the revised version we will expand §4 to include error bars across multiple runs, exact reward formulations, data split specifications, and statistical significance tests to substantiate the reported gains.","revision_made":"yes","referee_comment":"[Experiments] §4 (Experiments): The abstract reports a 16.7-point average gain and specific outperformance margins, yet provides no details on experimental controls, error bars, exact reward formulations, data splits, or statistical significance. These omissions prevent verification that the gains support the central claim rather than arising from uncontrolled factors."}],"tokens_in":1511,"tokens_out":428,"duration_ms":28735,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main move is repositioning VLMs as teachers that pull task rules into differentiable rewards, then use those to adapt a VGM reasoner online via lightweight LoRA at test time. This is positioned against prior VLM-as-solver work that relies on textual plans, which the authors say miss spatiotemporal details.\n\nIt does a reasonable job naming a practical limitation in current video reasoning setups and offering a test-time adaptation route that avoids full retraining. The reported 16.7-point average lift on VBVR-Bench and RULER-Bench over the solver baseline and best-of-N is the concrete result they highlight, and the comparable test-time cost claim makes the method sound deployable.\n\nThe soft spot is exactly the one the stress-test flags: turning VLM outputs into end-to-end differentiable rewards. Standard VLMs produce discrete or non-differentiable signals, so the gradient path from reward back to generated video latents needs an explicit construction. The abstract gives no equations or pseudocode for this step, and if the full paper does not supply one (embedding similarity, soft proxy, auxiliary head, etc.), the optimization loop reduces to something closer to non-differentiable search. Experimental controls, variance, and exact reward formulations are also thin in the provided text, which makes the size of the gains harder to evaluate.\n\nThis is for groups working on multimodal reasoning, test-time methods, or video generation. Readers who care about combining perception models with generators will see a usable direction. The work shows clear problem framing and an empirical hook, so it deserves peer review even though the differentiability detail will need tightening.","headline":"The core idea is using VLMs to generate differentiable rewards that drive test-time LoRA optimization on VGMs for video reasoning, with reported large gains, but the reward construction remains the critical unspecified step.","tokens_in":2504,"tokens_out":416,"would_cite":false,"duration_ms":33187,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"VLMs can teach video generation models to reason by turning perception into differentiable rewards optimized at test time.","keywords":["video reasoning","vision-language models","test-time optimization","video generation models","differentiable rewards","LoRA adaptation","VLM teacher"],"falsifier":"Running the method on a benchmark where VLM reward signals are deliberately noisy or inverted and checking whether performance still improves over the base VGM or drops below it.","tokens_in":2769,"feed_emoji":"🎥","tokens_out":627,"duration_ms":19887,"temperature":0.7,"pith_summary":"The paper argues that video generation models often fail at following task rules despite good visual quality, while vision-language models are weak at directly solving reasoning tasks but strong at perceiving whether constraints are met. It proposes shifting VLMs to the role of teachers that extract rules and create differentiable rewards to guide a lightweight adaptation of the video model during inference. This test-time optimization approach is shown to deliver substantial gains on symbolic and general video reasoning benchmarks compared to using VLMs as direct solvers or simple scaling methods.","feed_headline":"VLMs teach video models to reason with test-time rewards","feed_subtitle":"Differentiable rewards extracted from VLMs guide online LoRA adaptation, lifting average benchmark scores by 16.7 points.","key_machinery":"VLM-as-Teacher paradigm that converts perception of process constraints and goal achievement into differentiable rewards for test-time LoRA optimization of a video generation model.","core_discovery":"A VLM teacher extracts task-specific rules to formulate differentiable rewards, guiding a VGM Reasoner via test-time online optimization of a lightweight LoRA module. This strategy enables adaptive test-time optimization and extends the reasoning capabilities beyond the VGM's intrinsic boundaries.","pith_inferences":["If the reward formulation generalizes across tasks, it could reduce the need for task-specific fine-tuning of large video models.","The method suggests a broader pattern where perception models supervise generation models at inference time instead of competing with them as solvers.","Extending the approach might allow chaining multiple VLM teachers for multi-step or hierarchical reasoning without increasing model size."],"forward_implications":["Video reasoning performance rises by 16.7 points on average across symbolic and general-purpose benchmarks.","The VLM-as-Teacher approach outperforms both the VLM-as-Solver baseline and Best-of-N sampling by large margins at similar test-time cost.","Reasoning can be extended beyond a fixed VGM's training distribution through online reward-driven adaptation rather than prompt engineering alone.","The same teacher-reward loop can be applied to other generation-based reasoning tasks where perception is easier than generation."],"fun_headline_variants":["VLMs extract rules to create rewards for test-time VGM optimization","Test-time LoRA uses VLM rewards for video reasoning","VLM teachers enable test-time optimization of video models","Differentiable rewards guide VGM via test-time LoRA adaptation"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"VLMs must have accurate enough perception to correctly judge whether video generation steps satisfy task rules and reach the goal so that the resulting rewards actually steer the model toward correct solutions.","fun_headline_variants_meta":{"raw":{"variants":["VLMs extract rules to create rewards for test-time VGM optimization","Test-time LoRA uses VLM rewards for video reasoning","VLM teachers enable test-time optimization of video models","Differentiable rewards guide VGM via test-time LoRA adaptation"]},"model":"grok-4.3","cost_usd":0.003825,"raw_usage":{"total_tokens":2000,"prompt_tokens":725,"num_sources_used":0,"completion_tokens":68,"cost_in_usd_ticks":38249500,"prompt_tokens_details":{"text_tokens":725,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1207,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":725,"tokens_out":68,"duration_ms":13978,"temperature":1.0,"reasoning_tokens":1207,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T10:35:52.409054+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the method on a benchmark where VLM reward signals are deliberately noisy or inverted and checking whether performance still improves over the base VGM or drops below it.","supporting_citations":[],"review_version":2}