{"id":"54f0ea2c-de78-4aa5-ab31-cee79040ee22","arxiv_id":"2610.25763","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"TrajWAM predicts future object masks to guide one-step video generation, then conditions action prediction on the imagined future for efficient dynamic manipulation.","lead":"This paper presents TrajWAM, a robotic manipulation system that predicts future object motion to guide a video-generation model, then uses the imagined video to plan actions. It achieves state-of-the-art results on dynamic grasping tasks while running fast enough for closed-loop control.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Missing ablation: system not evaluated with ground-truth future masks, so the marginal benefit of trajectory guidance vs. mask prediction accuracy is unclear.","rationale":"The reader correctly identified the independent-motion assumption as a key limitation, which the paper acknowledges. My concern extends this: even for independently moving objects, the method relies on a motion predictor that could be inaccurate. The paper does not quantify how much performance degrades due to prediction errors. This is a load-bearing gap because the trajectory condition is the central innovation. However, the paper provides extensive ablations and real-world validation, so the concern is testable and does not invalidate the claims. The CONDITIONAL verdict remains appropriate, with an additional condition that the authors provide this ablation. I recommend no change to the reader's verdict.","tokens_in":19656,"tokens_out":3433,"duration_ms":37030,"concrete_test":"On the 10-task subset used in Table III(a), compare three conditions: (1) predicted future masks (current system), (2) ground-truth future masks from the dataset, and (3) all-zero masks (no guidance). Use the same trained checkpoint for all conditions except the mask sequence fed into the Trajectory ControlNet. If condition (2) outperforms (1) by a large margin (e.g., >10 percentage points SR), the mask predictor is a bottleneck. If (1) and (2) are comparable, the trajectory guidance itself is the key enabler and the predictor is sufficient.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that explicit trajectory guidance (predicted future masks) enables state-of-the-art performance is supported by ablation Table III(a), which compares Predicted masks to Static (repeat current mask) and Black (all-zero) conditions. However, this ablation does not compare against ground-truth future masks. Since the mask predictor is trained on the same distribution (DOMINO) and the entire pipeline from training through distillation uses predicted masks, it is unknown how much performance depends on the accuracy of the motion predictor versus the trajectory guidance mechanism itself. If ground-truth future masks lead to substantially higher success rates, then the current results may overstate the robustness of the trajectory guidance: performance could degrade significantly if the mask predictor fails on out-of-distribution objects or motion patterns. The paper acknowledges independent-object-motion as a limitation, but does not address the separate risk of mask prediction inaccuracy. Without this ablation, the reader cannot assess whether the method's gains come from the trajectory-conditioning design or from the specific motion predictor that was trained on the same benchmark. This gap is particularly load-bearing because the real-world evaluation also relies on SAM3 (asynchronous, 69.1ms latency) plus the same predictor; any failure of the predictor would cascade through the pipeline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper introduces TrajWAM, a world action model that predicts future object masks to explicitly guide video generation and action prediction for dynamic robotic manipulation. A lightweight Trajectory ControlNet steers a frozen video model toward trajectory-consistent futures, and a decoupled distillation scheme reduces inference to a single step while preserving the conditioning pathway. Experiments on the DOMINO benchmark (35 tasks, 3,500 episodes) and two real-world dynamic grasping setups show state-of-the-art success rates (43.54% on DOMINO, 77.9% on conveyor-belt grasping) with a latency of 108.5 ms, outperforming several VLA and WAM baselines.","tokens_in":20100,"tokens_out":2511,"duration_ms":26042,"significance":"If the reported results hold, TrajWAM demonstrates that explicit, object-centric trajectory conditioning can substantially improve the performance of world action models on dynamic manipulation, where scene-level forecasts often miss target motion. The decoupled distillation method is a practical contribution for deploying such models at control-relevant speeds. The work is timely given the growing interest in coupling prediction and control in robotics, and the extensive evaluations in both simulation and the real world strengthen the empirical case.","major_comments":[{"comment":"The central claim that explicit trajectory guidance (predicted future masks) enables state-of-the-art performance is supported by ablations that compare Predicted masks to Static (repeat current mask) and Black (all-zero masks). However, no comparison is made against ground-truth future masks. Since the mask predictor is trained on the same DOMINO distribution and the entire pipeline uses predicted masks, it is impossible to disentangle whether the gain comes from the trajectory-conditioning mechanism itself or from the specific learned predictor. If ground-truth future masks lead to substantially higher success rates, then the current results overstate the robustness of trajectory guidance and mask prediction errors would be a critical bottleneck. The authors should add an experiment with oracle future masks (e.g., simulator ground truth) on the same 10-task subset, report the results, and discuss any gap.","section":"Table III(a), Section IV-C2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The paper acknowledges that the method assumes target-object motion is independent of the robot's actions, and notes this as a limitation. However, this assumption is not treated as a load-bearing restriction in the design: it would invalidate the trajectory predictor in tasks such as pushing or deformation. The paper's own limitation statement is appropriate, but the evaluation protocol does not stress-test the boundary of this assumption. A more explicit discussion of when trajectory conditioning would need to be supplemented or replaced (e.g., through interaction-aware dynamics) would strengthen the claims about generalizability.","section":"Section V, Conclusion","authors_acknowledge":"However, the current experiments focus primarily on independently moving target objects; extending trajectory forecasting to action- dependent object dynamics remains an important direction toward more general dynamic manipulation.","authors_acknowledge_p":0.74}],"minor_comments":[{"comment":"The paper consistently writes 'TrajW AM' with a space between 'W' and 'AM' in the abstract and main text (e.g., 'TrajW AM achieves state-of-the-art performance'). This appears to be a typo; the intended name should be 'TrajWAM' as used in the method description. Please unify the notation.","section":"Abstract and Section I","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The caption references 'TrajW AM' (extra space) and the x-axis label in panel (b) says 'TrajWAM (ours)'. Please ensure consistency.","section":"Fig. 1 caption","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The task subsets for the ablations are listed, but the selection criteria for the 10-task subsets in Table III(a) and (c) could be stated more clearly. The current text in Appendix A.2 mentions stratification based on teacher policy success rates for the action-distillation set, but not for the other subsets.","section":"Appendix A, Table V","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The DMD objective in Eq. (7) uses a stop-gradient operation on the term 'sg[ˆy0 − Δ_DMD]'. The notation is clear, but a brief explanation of why the stop-gradient is needed (to prevent the generator from exploiting the score-model error) would help readers less familiar with DMD.","section":"Section III-C, Eq. (7)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"In the description of the mask conditioning: 'The resulting 384×320 frames are replicated across three channels and mapped from [0,1] to [−1,1] before video-VAE encoding.' It is unclear why three-channel replication is needed for a binary mask; please clarify whether this is an architectural requirement of the VAE encoder.","section":"Appendix B.2","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"The core idea is sensible: predict object-mask trajectories, use them to condition video generation, then feed the resulting video representation into an action model. The decoupled distillation pipeline—video backbone DMD, then ControlNet realignment, then action rollout distillation—is a pragmatic way to get one-step inference without breaking the conditioning interface. The empirical results on DOMINO (43.54% SR) and two real-world tasks (77.9% and 70.4%) are clearly ahead of baselines like InternVLA-A1.5 and Fast-WAM. That is real work, and the authors deserve credit for the integration and the evaluation effort.\n\nThe soft spot I keep coming back to is the missing ground-truth ablation. Table III(a) compares predicted future masks to static and black masks, but never to ground-truth future masks. Without that comparison, we don't know how much of the gain comes from the trajectory-guidance design versus the mask predictor's accuracy on the training distribution. If ground-truth masks would push SR much higher, then the system is fragile to predictor errors and the claimed advantage of explicit guidance is partly an artifact of a good but domain-specific motion predictor. The paper acknowledges the independent-motion limitation in the conclusion, but this mask-accuracy dependency is separate and not addressed.\n\nOther concerns are smaller in proportion. The real-world evaluation covers only two tasks and four object categories—respectable but narrow. No code or models are released, which limits reproducibility. The distillation hyperparameters are not fully ablated, though the comparison of distillation strategies (CD, DMD, rollout) is useful. The independent-motion assumption is a real restriction, but it is stated upfront.\n\nThis paper is for researchers working on dynamic manipulation, world models, or diffusion distillation for robotics. It deserves a serious referee because the core idea is solid and the experiments are otherwise thorough. The missing ablation is fixable in revision: adding a ground-truth-future-mask condition would clarify the claim. I would not cite it as-is because I want to see that ablation first. It is a serious paper that shows clear thinking about integration and deployment efficiency.","headline":"TrajWAM is a well-engineered system with strong empirical results, but the central claim about trajectory guidance is undercut by a missing ablation—no evaluation with ground-truth future masks—so we can't separate the effect of the guidance from the motion predictor's accuracy.","tokens_in":20627,"tokens_out":1491,"duration_ms":16514},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"TrajWAM uses predicted object masks to guide video generation for dynamic manipulation, achieving state-of-the-art success rates with efficient one-step inference.","keywords":["dynamic manipulation","world action model","trajectory guidance","video generation","diffusion distillation","robot grasping","object mask prediction"],"falsifier":"Evaluate TrajWAM on a pushing task where the robot must slide a deformable object to a target: if the trajectory predictor ignores the robot's contact forces, the future masks will diverge from reality and the grasping success rate should drop significantly compared to a method that models action-dependent dynamics.","tokens_in":21142,"feed_emoji":"🤖","tokens_out":2747,"duration_ms":28907,"temperature":0.7,"pith_summary":"Dynamic robotic manipulation requires anticipating moving targets and replanning quickly, but existing world action models (WAMs) either lack explicit object motion modeling or suffer from slow iterative denoising. TrajWAM introduces a trajectory-guidance mechanism: a motion predictor forecasts future masks of the target object, and a lightweight Trajectory ControlNet steers a pretrained video generator toward motion-consistent futures. The resulting future representation conditions an action expert to predict executable actions. To reduce latency while preserving this conditioning, the paper proposes a decoupled distillation strategy that compresses video generation, realigns the control adapter, and distills the action branch separately. The result is a system that outperforms prior methods on the DOMINO benchmark (43.54% success rate vs. 29.30% for the strongest baseline) and on real-world conveyor-belt and rotating-disc grasping (77.9% and 70.4% success rates, respectively), with an inference latency of 108.5 ms suitable for closed-loop control.","feed_headline":"Trajectory-guided vision boosts dynamic robot grasp success by 14 points","feed_subtitle":"A new world action model uses predicted object masks to guide video generation, enabling faster and more accurate interception of moving…","key_machinery":"The Trajectory ControlNet is the central component that bridges predicted object motion and video generation. It is a lightweight network (hidden width 512, versus the video backbone's 3,072) that takes the VAE-encoded future mask sequence and the video flow time as inputs, and injects layer-wise residuals into the frozen pretrained video model (Wan2.2-5B). This forces the generated future frames to be consistent with the predicted spatial trajectory of the target object. The decoupled distillation scheme is the second key mechanism: it separately compresses the video generator (via distribution matching distillation), realigns the Trajectory ControlNet to the compressed backbone (via latent reconstruction and an adversarial discriminator), and finally distills the action branch to produce accurate action chunks in one step while conditioned on the compressed future representation.","core_discovery":"TrajWAM demonstrates that explicitly grounding future visual imagination in predicted target-object motion significantly improves dynamic manipulation performance. The paper's central claim is that by predicting a sequence of future object masks from a short observation history, and injecting these masks into a pretrained video model via a Trajectory ControlNet, the system can generate future video representations that accurately reflect where the target will be. These trajectory-guided video representations then enable the action branch to predict action chunks that anticipate the object's motion, rather than reacting to outdated observations. Furthermore, through a decoupled distillation scheme—comprising distribution matching distillation for the video backbone, realignment of the trajectory adapter, and action rollout distillation—the full pipeline can be collapsed to one-step inference without sacrificing the benefits of trajectory conditioning. The result is a world action model that is both accurate and fast enough for real-time dynamic manipulation.","pith_inferences":["The assumption of independent object motion means the method will struggle in tasks where the robot's actions alter the target's trajectory (e.g., pushing, deformation). Extending the motion predictor to be action-conditional is a natural next step the paper acknowledges.","The trajectory predictor's reliance on a fixed third-person camera view limits applicability when the global view is occluded or when the robot arm obscures the target, which could be mitigated by multiple or moving cameras.","The real-world gains over baselines suggest that explicit trajectory guidance is more beneficial than implicit motion encoding (as in Fast-WAM) for high-speed grasping, potentially motivating a re-evaluation of existing WAM architectures for dynamic tasks.","The distillation pipeline's three-stage decoupling may be transferable to other conditional video-action models, such as those for surgical robotics or autonomous driving, where low latency and accurate prediction of moving agents are also critical."],"forward_implications":["Dynamic manipulation robots can operate at higher speeds and with greater reliability because the system explicitly anticipates where the target will be, reducing missed grasps due to latency.","The trajectory-guidance mechanism can be integrated into other world action models, providing a general way to inject object-centric motion priors into video-conditioned action policies.","The decoupled distillation strategy offers a template for accelerating other conditional generative pipelines where multiple modalities (video and action) must be compressed without losing conditioning accuracy.","The approach may enable robots to intercept faster-moving objects or operate in cluttered environments where reactive policies fail, as shown by the high-speed gains in real-world experiments.","The framework's robustness to different mask sources (ground truth, SAM3, Grounded SAM2) suggests that it can be deployed with off-the-shelf segmentation, reducing the engineering burden for new tasks."],"supporting_citations":[{"why":"Provides the DOMINO benchmark and the PUMA baseline used for simulation evaluation.","marker":"[15]"},{"why":"Supplies the Wan2.2-5B video generation model that TrajWAM initializes and fine-tunes.","marker":"[32]"},{"why":"Introduced Fast-WAM which serves as both a baseline and the initial checkpoint for TrajWAM's video and action branches.","marker":"[7]"},{"why":"Proposes the dense-to-sparse trajectory guidance mechanism that inspires the Trajectory ControlNet design.","marker":"[22]"},{"why":"Gives the distribution matching distillation (DMD) method used for video backbone distillation in this work.","marker":"[33]"},{"why":"Describes the π0 vision-language-action flow model used as a baseline for comparison.","marker":"[13]"},{"why":"Provides consistency distillation (CD) which is compared as an alternative action distillation strategy.","marker":"[24]"}],"fun_headline_variants":["TrajWAM uses predicted object masks for real-time dynamic manipulation","One-step video-action inference via trajectory-guided world model","Predicting future masks guides video generation for fast robot grasping","TrajWAM achieves SOTA on dynamic object manipulation tasks","Trajectory-guided anticipation boosts robot interception speed"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The system assumes that the target object's future trajectory is independent of the robot's actions, so the predicted masks remain correct regardless of what the robot does.","fun_headline_variants_meta":{"raw":{"variants":["TrajWAM uses predicted object masks for real-time dynamic manipulation","One-step video-action inference via trajectory-guided world model","Predicting future masks guides video generation for fast robot grasping","TrajWAM achieves SOTA on dynamic object manipulation tasks","Trajectory-guided anticipation boosts robot interception speed"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000392,"raw_usage":{"cost":0.0003920609,"is_byok":false,"cost_details":{"upstream_inference_cost":0.0003920609,"upstream_inference_prompt_cost":9.113258e-05,"upstream_inference_completions_cost":0.00030092832},"total_tokens":2659,"prompt_tokens":1003,"completion_tokens":1656,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":1574}},"tokens_in":1003,"tokens_out":1656,"cost_source":"openrouter_usage","duration_ms":16255,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":1574,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-20T14:27:31.766777+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Evaluate TrajWAM on a pushing task where the robot must slide a deformable object to a target: if the trajectory predictor ignores the robot's contact forces, the future masks will diverge from reality and the grasping success rate should drop significantly compared to a method that models action-dependent dynamics.","supporting_citations":[{"cited_title":"Towards generalizable robotic manipulation in dynamic environments,","cited_arxiv_id":null,"evidence_quote":"Provides the DOMINO benchmark and the PUMA baseline used for simulation evaluation."},{"cited_title":"Wan: Open and Advanced Large-Scale Video Generative Models","cited_arxiv_id":"2503.20314","evidence_quote":"Supplies the Wan2.2-5B video generation model that TrajWAM initializes and fine-tunes."},{"cited_title":"Fast-WAM: Do World Action Models Need Test-time Future Imagination?","cited_arxiv_id":"2603.16666","evidence_quote":"Introduced Fast-WAM which serves as both a baseline and the initial checkpoint for TrajWAM's video and action branches."},{"cited_title":"MagicMotion: Controllable video generation with dense-to-sparse trajectory guidance,","cited_arxiv_id":null,"evidence_quote":"Proposes the dense-to-sparse trajectory guidance mechanism that inspires the Trajectory ControlNet design."},{"cited_title":"One-step diffusion with distribution matching distillation,","cited_arxiv_id":null,"evidence_quote":"Gives the distribution matching distillation (DMD) method used for video backbone distillation in this work."},{"cited_title":"$\\pi_0$: A Vision-Language-Action Flow Model for General Robot Control","cited_arxiv_id":"2410.24164","evidence_quote":"Describes the π0 vision-language-action flow model used as a baseline for comparison."},{"cited_title":"Consistency models,","cited_arxiv_id":null,"evidence_quote":"Provides consistency distillation (CD) which is compared as an alternative action distillation strategy."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}