{"id":"4a4d25f2-0d0b-45c5-a406-b5ef5a8b9f3e","arxiv_id":"2602.19313","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The log-probability a VLM assigns to 'True' for 'does this video prefix complete the task?' is used as a zero-shot dense progress reward that outperforms GVL on open-source models.","lead":"TOPReward turns a video-language model's internal 'True'-token probability into a dense progress reward for robot manipulation, with zero reward-model training. It reportedly beats the GVL baseline on open-source VLMs over 130+ tasks, but key benchmark data and control analyses are not yet public.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VOC measures monotonicity within an episode, not absolute completion; the paper's own success-detection experiment shows TOPReward's VOC is nearly identical on failed and successful trajectories, so the central claim that the signal tracks genuine task progress is not yet established.","rationale":"The reader's weakest assumption identified the time-index confound. My independent analysis agrees and further strengthens it: the paper's own Section 5.2 is the decisive evidence. TOPReward's VOC is 0.946 on failed trajectories vs 0.943 on successful ones, yet success detection AUC is only 0.654 for Qwen3-VL. This moderate AUC, combined with the identical VOC, shows that the ordering-based metric is insensitive to completion and that the raw log-probability signal is only weakly separating success from failure at the trajectory level. Since the whole point of a temporal value function is to distinguish progress (monotonic increase) from stalled/failed behavior, a signal that fails to separate completed and incomplete trajectories is not sufficient evidence for the central claim. Moreover, the abstract says 'not explained by time index alone,' but no such analysis appears: no partial correlation controlling for time, no time-only baseline. The success-detection experiment uses average log-prob over the last 3 frames, not the normalized per-episode VOC, so one cannot recover from the reported mean VOC alone whether final scores separate success/failure. The check I propose is exactly the missing analysis. My verdict remains CONDITIONAL: the gaps are addressable and the method may be sound, but as written the central numeric claims are not substantiated. This is not a rejection: the success-detection result (0.654) is above chance and on Gemini it matches a strong baseline, but it is not a demonstration of the claimed 0.947 progress understanding. I also note the restricted benchmark and unavailable code further prevent independent verification, which the reader mentioned.","tokens_in":17987,"tokens_out":1814,"duration_ms":17481,"concrete_test":"On the existing failure split: compute Spearman's rank correlation between TOPReward's raw per-episode final log-probabilities and ground-truth success labels (or report ROC-AUC using the raw log-prob as the score). If AUC is not significantly above the 0.654 reported, the signal is not measuring completion. Additionally, run a permutation control: for each episode, shuffle the frame order but keep the same frames, evaluate TOPReward on random permutations; if a time-correlated artifact explains VOC, permuted orderings would still yield high rank correlation. The key test: on the 156 failure episodes, does TOPReward assign lower final scores than on the 497 successful episodes, and by how much?","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's headline claim is that TOPReward's log-probability signal tracks genuine task progress (0.947 VOC on Qwen3-VL). But VOC (Eq. 4) is Spearman rank correlation between predicted scores and frame time. Any signal that increases with video length—regardless of task content—achieves high VOC. Section 5.2 explicitly demonstrates this: on failed trajectories that rise and plateau, TOPReward's mean VOC is 0.946, virtually identical to the 0.943 on successful trajectories. The failure split is exactly the control the reader requested: the signal is not explained by time index alone. On these failures, the robot makes progress early and then moves randomly; the ground-truth progress plateaus, but because VOC only checks ordering, both a correct plateauing signal and a time-correlated artifact would score ~0.95. A signal that cannot distinguish success from failure at the trajectory level cannot be a reliable progress reward. The paper does not report the promised analysis of whether VOC is explained by time index alone—no partial-correlation, no baseline using frame index as the predictor, no comparison of log-prob against a time-only control. The token-probability curve (Fig. 4) may be genuinely content-driven, but the evidence in the text does not exclude the simpler explanation: the VLM's P(True) increases with the amount of video evidence, not with task-specific progress.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"TOPReward proposes a training-free reward signal for robotic manipulation: given a video prefix and a language instruction, it logs the probability that a VLM assigns to the affirmative token 'True' in response to a completion query, and uses this log-probability (after per-episode min-max normalization and a clipped incremental transform) as a dense progress reward. The paper introduces ManiRewardBench, a real-world manipulation benchmark with subtask-level temporal annotations, and reports that TOPReward achieves 0.947 mean VOC with Qwen3-VL-8B on this benchmark, substantially outperforming the GVL baseline on open-source models. It also reports success-detection ROC-AUC results and real-world advantage-weighted behavior cloning experiments on six SO-100 tasks, where TOP-AWR outperforms BC.","tokens_in":18336,"tokens_out":3491,"duration_ms":33880,"significance":"If the core claim is correct, TOPReward would be a valuable and simple result: extracting dense progress supervision from frozen VLMs without any training could remove a major bottleneck in robot learning. The paper's strengths include the breadth of the evaluation (130+ tasks, multiple embodiments), the release of a new benchmark with fine-grained subtask annotations, and the downstream validation via success detection and behavior-cloning fine-tuning. The method itself is transparent and easy to implement. However, the central metric (VOC) and the absence of a direct time-index control leave the main claim—that the signal reflects task-specific progress rather than a generic temporal artifact—incompletely supported. The success-detection experiments provide some evidence of content sensitivity, but the 0.654 AUC on Qwen3-VL is modest and does not by itself rescue the VOC-based progress-estimation claim.","major_comments":[{"comment":"The headline metric VOC is rank-correlation of predicted scores with frame time. The paper's own Section 5.2 states that VOC is virtually identical for failed and successful trajectories (0.946 vs. 0.943) and that a trajectory which rises and then plateaus can still achieve high VOC. This directly undermines the interpretation of the reported 0.947 VOC as evidence that the signal 'tracks genuine task progress.' A time-only control (e.g., using frame index as the predictor, a partial-correlation analysis, or showing the log-probability curve on the failure split after plateau) is needed to support the abstract's assertion that the reward is 'not explained by time index alone.' Without such an analysis, the central claim is not established.","section":"§5.1, Eq. (4), and §5.2"},{"comment":"The affirmative token 'True' was selected post hoc because it showed the largest success–failure separation on the evaluation benchmark (Figure 8). This is a free parameter chosen using the same episodes on which the method is then evaluated. Such selection can inflate the reported performance and needs to be justified with a held-out selection procedure or a correction. At minimum, the authors should report the VOC and success-detection results for a few plausible alternative tokens (e.g., 'Yes', 'Complete') to demonstrate robustness to this choice.","section":"§3.1 and Appendix B"},{"comment":"The abstract promises 'additional analyses show that the reward is sensitive to the specified instruction and is not explained by time index alone,' but the full text contains no such analysis. The only relevant result is the success-detection ROC-AUC of 0.654 on Qwen3-VL-8B, which is weakly above chance and uses a different signal (average log-probability over the last three frames) rather than the dense per-prefix reward used for VOC. Please either add the promised time-index analysis or remove the claim; a concrete control would strengthen the paper considerably.","section":"Abstract and §5.2"}],"minor_comments":[{"comment":"The table formatting is broken: numbers run together (e.g., '1 23', '1.33 12.33', '5.676.33'), making it hard to read the per-method scores. Please align columns and use separate columns for Pretrained, BC, and TOP-AWR.","section":"Table 4"},{"comment":"The chat-template ablation is informative, but the mean VOCs reported in Table 5 (0.945 and 0.652 for Qwen and Molmo) do not match the mean values in Table 2 (0.947 and 0.642). Please check the averaging procedure and report error bars or standard deviations.","section":"§5.4 and Table 5"},{"comment":"The token-selection plot should include error bars or confidence intervals, and the number of episodes used to compute the separation should be stated. Currently it is unclear how stable the 'True' preference is across tasks and models.","section":"Appendix B, Figure 8"},{"comment":"ManiRewardBench is described as restricted-access with a 'controlled evaluation protocol.' This limits reproducibility. Please clarify what the protocol is and whether an evaluation server or benchmark release is planned.","section":"§4"}],"recommendation":"major_revision","confidential_remarks":"The paper makes a strong and potentially useful claim, but the evidence as presented is not yet convincing. The VOC metric, combined with the post hoc token selection, creates a real risk that the reported gains reflect a temporal confound rather than task understanding. The authors should be asked to add a time-index control, show that the signal distinguishes success from failure in a graded sense, and justify or validate the choice of 'True' on held-out data. If these concerns are addressed, the paper could be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is genuinely new and well motivated: instead of asking a VLM to generate a progress number, read the log-probability of a completion token from trajectory prefixes and use that as a dense zero-shot reward. That is a concrete, clean reformulation that GVL and SuccessVQA-style methods don't cover, and it makes good use of the fact that open-source VLMs often fail at numeric generation while retaining video understanding. The real-robot AWR results, with consistent improvements over behavior cloning across six SO-100 tasks, are the most convincing part of the paper; they give me more confidence than any VOC number that the signal carries task-relevant information. The benchmark work is also substantial, with careful subtask annotations across multiple platforms.\n\nThe soft spots are real, and they all point at the same question: what exactly is the reward tracking? The headline metric, VOC, is Spearman correlation between predicted scores and frame time, so any signal that grows with video length passes. The paper actually supplies a nice failure-mode analysis showing that plateauing trajectories can score VOC ≥ 0.85, and then reports that TOPReward's mean VOC on failed trajectories (0.946) is virtually identical to successful ones (0.943). That is close to a direct admission that VOC is not measuring completion. The promised 'not explained by time index alone' analysis is absent; there is no partial correlation, no frame-index baseline, no comparison on the failure split. The success-detection result, 0.654 ROC-AUC on Qwen3-VL, is better than GVL's 0.519 but still suggests the log-probability signal is only weakly separating success from failure.\n\nThere are also smaller issues: the 'True' token was selected post hoc on benchmark episodes, the trained reward-model comparison named in the abstract never appears, and ManiRewardBench access is restricted, which limits independent verification. None of these are fatal. The method is training-free, the underlying signal is a real VLM log-probability rather than a fitted target, and the authors are unusually clear about the VOC metric's limitations — they identify the failure mode and then try to evaluate around it, even if their evaluation doesn't fully answer the question.\n\nThe paper deserves serious refereeing. The idea is important if it holds; the gaps are specific, addressable, and clearly stated. A serious referee could push for the time-control analysis, for a calibrated comparison of absolute scores on success versus failure, and for open benchmark access. I would send it to review, with the expectation of substantial revision rather than rejection.","headline":"A clean, useful idea with an evaluation that doesn't yet separate progress understanding from time correlation — worth referee time, but the headline VOC numbers should be treated as unproven until the promised control appears.","tokens_in":18848,"tokens_out":1261,"would_cite":true,"duration_ms":11519,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Token probabilities yield a zero-shot robot progress reward, hitting 0.947 rank correlation with time order on 130+ tasks.","keywords":["token probabilities","zero-shot reward","video-language models","progress estimation","robot manipulation","temporal value function","ManiRewardBench","behavior cloning"],"falsifier":"Present a video of a robot executing a task that stalls after an early partial success (e.g., grasping then dropping), and observe whether the log-probability of 'True' continues to increase as more stalled frames are added; if it does, the reward is tracking time rather than task content. A simpler check is to reverse the frame order: a progress reward that still rises with the number of frames would not be tracking progress.","tokens_in":17884,"feed_emoji":"🤖","tokens_out":14086,"duration_ms":105269,"temperature":0.7,"pith_summary":"This paper claims that a pretrained video-language model already contains enough latent understanding of task completion to serve as a dense reward signal for robotic manipulation, if the reward is read from the model's internal token probabilities rather than from its generated text. The proposed method, TOPReward, measures the log-probability of the affirmative token 'True' in response to a completion query about a video prefix and an instruction, and normalizes this signal per episode to produce a progress curve. Across more than 130 real-world manipulation tasks on four robot platforms, the method achieves high agreement between predicted progress order and time order, and it supports downstream uses such as success detection and advantage-weighted behavior cloning. If correct, this would remove a major bottleneck in robot learning—the need for hand-crafted or trained reward functions—by showing that reward modeling is an emergent capability of off-the-shelf video-language models.","feed_headline":"Probing token probabilities yields zero-shot robot progress","feed_subtitle":"Training-free rewards for 130+ real-world manipulation tasks, using only the model's internal completion-token probability.","key_machinery":"The key machinery is the token-probability reward: for each video prefix, the method constructs a prompt that asks whether the trajectory completes an instruction, and takes the logarithm of the model's probability for the affirmative token 'True' as the reward. This converts the VLM's implicit world knowledge into a scalar that grows as evidence for completion accumulates; min-max normalization per episode yields a progress curve in [0,1], and increments can be clipped to form per-step dense rewards for policy learning. The method deliberately avoids chat templates, which the paper finds degrade performance, and relies on the raw next-token prediction distribution.","core_discovery":"The central discovery is that the log-probability of a binary completion token—'True'—computed from a video-language model's next-token distribution over a video prefix and an instruction is a valid dense progress signal for robotic manipulation, without any fine-tuning. The paper argues that open-source VLMs fail at progress estimation not because they lack temporal understanding but because their textual output of numerical values is unstable; by probing the model's internal token distribution instead, TOPReward obtains a smoothly increasing reward as the task nears completion. On the newly introduced ManiRewardBench benchmark—130+ tasks across Franka, SO-100/101, and single-arm/bimanual Y","pith_inferences":["A direct control experiment—fixing the number of frames while shuffling their content, or presenting reversed trajectories—would isolate whether the reward tracks task content or merely video length; the paper asserts but does not show this analysis.","TOPReward could be extended to long-horizon tasks by decomposing the instruction into subtask-level completion queries, trading extra forward passes for finer temporal resolution.","Using multiple affirmative tokens or contrasting token probabilities (e.g., True vs. False) might improve calibration and robustness across different VLM backbones, an avenue the paper leaves open."],"forward_implications":["Reward engineering for robotic manipulation can be replaced by a single completion-token query on an off-the-shelf video-language model, requiring no task-specific demonstrations or reward-model training.","Because the reward is instruction-conditioned, the same model supplies progress feedback across a wide range of tasks and embodiments, making language-conditioned reinforcement learning more accessible.","The completion-token probability doubles as a success detector, enabling automatic dataset filtering and offline weighted behavior cloning without a separately trained classifier.","The method's performance is bounded by the backbone video-language model's visual understanding, so advances in video VLMs should lift reward quality directly."],"fun_headline_variants":["VLM token probabilities give zero-shot robot progress","Probe one token for dense robot manipulation rewards","Training-free robot rewards from VLM token likelihoods","TOPReward: token likelihoods replace trained reward models","Robot task progress from VLM's internal token odds"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The evaluation assumes that a high rank correlation between TOPReward's predicted scores and frame time (VOC) demonstrates genuine understanding of task progress, rather than the trivial tendency for any completion statement to become more plausible as more frames are shown.","fun_headline_variants_meta":{"raw":{"variants":["VLM token probabilities give zero-shot robot progress","Probe one token for dense robot manipulation rewards","Training-free robot rewards from VLM token likelihoods","TOPReward: token likelihoods replace trained reward models","Robot task progress from VLM's internal token odds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000289,"raw_usage":{"total_tokens":1541,"prompt_tokens":764,"completion_tokens":777,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":701}},"tokens_in":508,"tokens_out":777,"duration_ms":7063,"temperature":1.0,"reasoning_tokens":701,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T21:39:01.233999+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Present a video of a robot executing a task that stalls after an early partial success (e.g., grasping then dropping), and observe whether the log-probability of 'True' continues to increase as more stalled frames are added; if it does, the reward is tracking time rather than task content. A simpler check is to reverse the frame order: a progress reward that still rises with the number of frames would not be tracking progress.","supporting_citations":[],"review_version":1}