{"id":"6ea97a44-89d0-4d5a-aeac-aa11cf3efb28","arxiv_id":"2610.12407","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A decoder-free JEPA world action model with a bidirectional transformer trained through forward, backward, inverse, and policy modes, whose diffusion-steering MPC plans in the policy's noise space and improves closed-loop control.","lead":"LeWAM is a robot control model that learns a compact latent picture of the world without reconstructing images, and uses that latent for predicting, acting, and planning. The paper shows that planning by steering a diffusion policy's noise, rather than sampling raw actions, gives much better closed-loop robot task success.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Planning contribution rests on an unvalidated latent-distance proxy for task success; the goal is retrieved from the model's own latent, so DS success may reflect policy-prior selection rather than goal-directed planning.","rationale":"The reader identifies the latent-distance proxy as the weakest assumption; I agree. The architecture and acting results are supported by multi-seed closed-loop evaluations and mode ablations, and the projection ablation in Table 4 is a genuine falsifiable check. But the planning benefit is the least supported of the three abstract claims: because the same latent space is used for goal retrieval and for scoring, the planner could select actions that merely keep rollouts consistent with the model's own representation, while the proposal distribution is simultaneously changed from raw actions to the flow head's typical set. A control that holds proposals fixed while changing the goal, plus a direct rank-correlation check between Eq. 1 and environment success, would separate goal-directed planning from policy-prior selection. The lack of tabulated per-task numbers and the absence of a best-of-N-versus-iterative-update baseline weaken verification further, but the proxy validity is the one condition on which the entire planning claim rests. This does not overturn the paper; it confirms the CONDITIONAL verdict already given.","tokens_in":14902,"tokens_out":9182,"duration_ms":132769,"concrete_test":"On PushT and Lift/Can, fix the policy head, budget (64x3), and projection, and compare DS-CEM goals: (a) the Appendix D retrieved goal, (b) a fixed goal latent from the final successful demonstration frame, (c) a random goal latent from another demonstration. Also, for a fixed set of start latents, score 64 policy-prior chunks by Eq. 1 and by actual simulator success and report the Spearman rank correlation. If closed-loop success is insensitive to goal choice, or the rank correlation is not significantly positive, Eq. 1 is not steering behavior and the planning claim needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Claim (3) requires that minimizing ||f(z_t,a)-z_g||_2 in Eq. 1 ranks action chunks by task progress. That equivalence is never checked. Section 4.1 imports the proxy from DINO-WM/LeWM; Section 5.4 defines z_g as the latent 25 steps after the demonstration frame nearest the current latent (Appendix D); Figure 5 reports only aggregate closed-loop success, not the goal-ranking quality. Since the same encoder produces both the retrieved goal and the scored final latent, ranking can be self-consistent without tracking true task state, and because DS samples only through the flow head g(epsilon; z), gains may come from staying on the policy's typical set rather than from Eq. 1. Table 4 makes this salient: removing the spherical projection changes Can from 78.8% to 23.0%. The authors acknowledge needing a goal at inference time, but not the validity of the proxy; that is the unresolved load-bearing step.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper proposes LeWAM, a decoder-free JEPA-based world action model in which a bidirectional transformer is trained through four modes—forward, backward, inverse, and policy prediction—over a single 384-dimensional latent produced by a from-scratch ViT encoder with SIGReg regularization. The authors claim three benefits: (1) linear probes read robot and object state from the LeWAM latent better than from LeWM while ignoring distractors; (2) the integrated flow-matching policy matches a reference behavioral-cloning flow policy in closed-loop evaluation while also serving as a dynamics model; and (3) sampling-based MPC over the policy head's noise space (diffusion steering) improves closed-loop planning compared with action-space MPC, which collapses on the small human-demonstration datasets used here. The evaluation covers PushT and five robomimic tasks with added visual distractors, using parameter-matched reimplementations of Dreamer, LeWM, a reconstruction-based WAM, and a flow-matching policy reference.","tokens_in":15119,"tokens_out":9063,"duration_ms":67387,"significance":"If the claims hold, the paper is a useful step toward reconstruction-free world action models: it shows that a single backbone can provide forward, backward, inverse, and policy prediction, that the resulting latent supports linear readout of task state while being robust to distractors, and that an inference-only change to the MPC decision variable can improve closed-loop success. The strengths are genuine and should be credited: multi-seed closed-loop evaluation (5 training seeds × 50 evaluation seeds), parameter-matched baselines, public tasks and demonstrations, an ablation table for the four training modes, and an ablation of the spherical projection step. The headline planning claim, however, currently rests on figures rather than auditable numeric tables, and the latent-distance goal proxy that Eq. (1) relies on is not validated against any task-reward signal; both are fixable within the scope of the manuscript.","major_comments":[{"comment":"The planning claim depends on the assumption that the Euclidean latent distance in Eq. (1) ranks action chunks by task progress, but this equivalence is never tested. The goal latent is obtained by encoding the observation 25 steps after the demonstration frame nearest the current latent, in the model's own latent space, so the ranking can be internally self-consistent without tracking the true block or end-effector state. This is load-bearing because the projection ablation (Table 4) shows a large effect of staying on the sampler's typical set (Can: 78.8% with the sqrt(A) shell versus 23.0% without), so the DS gain could partly or wholly come from policy-prior selection rather than from goal-directed search through Eq. (1). Please add (a) a correlation study between Eq. (1) scores and task-state distance or success on held-out starts, (b) a DS-CEM control that scores candidates by ground-truth task-state distance to the goal, and (c) a sensitivity check on the 25-step retrieval horizon. The paper acknowledges that a goal is needed at inference time (Section 6), but it does not acknowledge or address the validity of the goal proxy itself.","section":"Section 4.1, Eq. (1); Section 5.4; Appendix D","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The headline closed-loop planning result is presented only graphically, with no numeric table giving per-task, per-planner means and error bars. The text's quantitative claims—that action-space MPC collapses every model to 3–8% success and that sampling in noise space gives both LeWAM and RecWAM a performance gain—cannot be verified from the manuscript as written. Appendix A points to per-task results in Figure 6, but again without a numeric table or seed-level statistics. Please add a table reporting means and standard deviations over the 5 training seeds and 50 evaluation seeds for each policy, each action-space MPC variant (CEM, MPPI, CMA-ES), and each diffusion-steering variant on every task.","section":"Section 5.4; Figures 5 and 6","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The paper does not separate the benefit of iterative planning from the benefit of best-of-N proposal ranking over the policy prior. Section 4.3 states that the first iteration of every DS planner is best-of-N ranking with N=S over projected noise, yet no experiment compares this one-iteration procedure against the full I=3 DS-CEM. Without this control, the improvement attributed to MPC-style iterative refinement is not distinguished from the simpler explanation that sampling from the policy's typical set and ranking by a self-consistent latent distance is already sufficient for most of the gain. Please report I=1 versus I=3 DS-CEM, ideally matched in total proposal count, for LeWAM and RecWAM.","section":"Section 4.3; Algorithm 1","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The sentence \"The search distribution stays Euclidean; only proposals are projected\" appears twice in immediate succession; one occurrence should be removed.","section":"Section 4.4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The notation in Eq. (2) is confusing: the update is written as z_hat_h = shift(z_hat_{h-1}, z_hat_h), where the same symbol z_hat_h appears on both sides. Use a temporary variable or a different subscript to make the recurrence clear.","section":"Section 4.3, Eq. (2)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The MSE column for distractors is labelled \"Distractors↑\", but higher MSE means the distractor state is not encoded; the arrow notation is counterintuitive and should be explained in the caption.","section":"Table 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The statement that the latent \"ignores\" visual distractors is stronger than what the evidence supports; the linear probe results show that distractor state is not linearly decodable, and the qualitative decoder rollouts are suggestive but not a quantitative invariance measure. Please qualify this as \"not linearly decodable\" or add a nonlinear-probe control.","section":"Section 5.3 and Conclusion","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The \"No policy\" row is confounded by construction: that variant has no policy head, so its DS-CEM column steers the inverse-trained flow head, and its near-zero DS-CEM success shows that diffusion steering requires a policy head rather than that the policy mode improves the latent representation. The conclusion that \"all modes add value\" should be reworded to acknowledge this.","section":"Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"For Dreamer, action-space MPC is scored by pixel error of the decoded rollout rather than by the latent distance of Eq. (1); the main text should state this difference when comparing \"action-space MPC\" across models, since the comparison is not a pure ablation of the decision variable.","section":"Appendix D","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Here's my quick read. LeWAM is worth taking seriously. It isn't a rebranding: a bidirectional transformer on a LeWM-style JEPA latent, trained end-to-end with forward, backward, inverse, and policy modes on every batch, giving one model both a dynamics head and a flow-matching action head. The evaluation is properly structured—five training seeds, fifty eval seeds, parameter-matched baselines, public demonstration data, and a mode ablation. The acting claim in b2 holds up: LeWAM roughly matches the standalone flow-matching policy on the same encoder. The probe results are also credible: task-state R2 goes up and distractor R2 stays near zero. The no-policy ablation kills DS-CEM, which is what you'd expect if the planner is genuinely decoding through the policy head. The LeWM baseline is disclosed as frozen-LeWM plus their action head, and the DSRL/Golden Ticket lineage for noise-space search is credited. Fine so far.\n\nThe planning claim is the weak point, and the stress-test note is on target. Eq. 1 scores chunks by Euclidean distance to a goal latent retrieved from the model's own encoder; the same forward model both rolls out and scores, and the paper never checks that this proxy ranks chunks by task progress. Because the same encoder produces the goal and the imagined final latent, the ranking can be self-consistent without tracking true task state. The projection ablation makes the worry concrete: removing the sphere projection drops Can from 78.8% to 23.0%, a much larger effect than you'd expect from a temperature fix. There is also no table for Figure 5, and the design doesn't isolate one-iteration best-of-N proposal ranking from iterative MPC—the first DS iteration is already best-of-N. A random-goal or fixed-goal control, plus a numeric per-task table, would let a reader attribute the gain. The authors do acknowledge planners are slow and need a goal at inference (b123), but they don't acknowledge the proxy itself as the unresolved step.\n\nMinor: no code or weights.\n\nBottom line: this is a serious, carefully run paper with one central empirical claim that is under-supported. I'd send it to review; the architecture and acting results justify referee time, and the planning section can be fixed with targeted experiments. I'd want goal-proxy validation and a numeric table before signing off. Would cite it.","headline":"A genuinely new decoder-free JEPA action model with careful multi-seed evaluation; diffusion-steering planning works empirically, but the goal-latent proxy is not yet shown to be doing the work.","tokens_in":15640,"tokens_out":5980,"duration_ms":45380},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"This paper proposes LeWAM, a decoder-free JEPA transformer that predicts forward, backward and inverse dynamics plus policy actions, and shows that model-predictive control improves when it searches the policy head's noise space instead…","keywords":["JEPA","world action models","model predictive control","diffusion steering","flow matching","latent dynamics","robot manipulation","visual distractors"],"falsifier":"Score DS-CEM's proposed plans two ways on identical rollouts, by the paper's latent-distance cost $\\|\\hat{f}(z_t,a)-z_g\\|_2$ (Eq. 1) and by actual simulator success at reaching the target pose, and compare the rankings of the 192 candidates in a typical planning step; if the rankings disagree, or if noise-space planning stops beating action-space planning once a reward-based cost replaces the proxy, then the latent-distance proxy, not diffusion steering, carries the reported closed-loop gains.","tokens_in":16302,"feed_emoji":"🤖","tokens_out":15127,"duration_ms":110611,"temperature":0.7,"pith_summary":"LeWAM tries to give robots a single world action model that needs no pixel decoder: one bidirectional transformer, trained on a joint-embedding-predictive (JEPA) latent, that at once predicts future latents, back-predicts earlier latents, infers the actions bridging two observed states, and emits the next action chunk. The paper reports three payoffs. First, linear probes read robot and object state from LeWAM's latent more accurately than from the forward-only JEPA baseline, while visual distractors leave almost no trace in the latent. Second, acting with the learned flow-matching policy matches a standalone behavior-cloning policy at matched model size, while the same model still provides a dynamics model. Third, closed-loop model-predictive control improves when the planner searches the policy head's noise space, the Gaussian draw the flow sampler integrates, rather than raw action chunks; the paper's diagnosis is that sampling raw actions lets MPC exploit dynamics-model inaccuracies and optimize for hallucinated futures that fail in the simulator.","feed_headline":"Planning in the robot policy's noise space beats raw-action MPC","feed_subtitle":"A single JEPA transformer learns dynamics and policy from pixels; planning steers diffusion noise, not raw actions.","key_machinery":"Three mechanisms carry the argument. The latent: each frame becomes a single 384-dimensional vector from a ViT-tiny encoder trained from scratch, pushed toward an isotropic Gaussian by SIGReg, with no decoder, no tokenizer and no EMA target network; this one vector is what the probes read, the dynamics regress, and the planner compares. The backbone: a depth-8 bidirectional transformer reads the context latents and action tokens, and the four training modes are different maskings of the same input window - the final frame masked for forward dynamics, the first frame for backward dynamics, the target action chunk replaced by a noisy flow sample for inverse dynamics, and both the final frame and recent actions masked for policy prediction. The planner: the rectified-flow head is a deterministic map $a = g(\\epsilon; z_t)$ from a Gaussian draw to an action chunk, and diffusion steering moves the MPC decision variable to $\\epsilon$, projecting every proposal onto the sphere of radius $\\sqrt{A}$ before decoding so the search stays on the directions the head was trained to produce.","core_discovery":"Stated on the paper's own terms, the central claim is that a decoder-free JEPA latent trained end-to-end through four masked modes of one bidirectional transformer is simultaneously a cleaner state representation, a deployable policy, and a forward model good enough to plan with. The paper's argument is that reconstruction-based world action models such as Cosmos Policy force the latent to encode everything a pixel decoder could render, wasting capacity on distractors, while forward-only JEPA models such as LeWM offer no policy head; LeWAM couples LeWM's regularized latent with the UWM-style multi-mode skeleton and adds backward dynamics. On six manipulation tasks with visual distractors, the reported results are: LeWAM's latent beats LeWM on task-state probes (average R-squared of 0.80 vs 0.62) and slightly exceeds the reconstruction-based RecWAM (0.78), while reading essentially no distractor state (0.00 vs 0.73); its closed-loop acting holds the level of the flow-matching policy reference; and every action-space planner collapses to 3-8% closed-loop success, whereas the same planners steered in the policy's noise space improve on the policy itself.","pith_inferences":["A test the authors do not run: vary the number of distractors or their motion and track distractor-probe readout and closed-loop success; the near-zero distractor readout in Table 2 predicts graceful degradation under clutter, which is the property that would matter most on real-world data.","The paper's own explanation, that imperfect dynamics under small data make action-space search exploit model errors, predicts that noise-space planning's advantage shrinks as the dataset grows; a data-scaling curve of DS-CEM versus action-space CEM would separate the steering mechanism from mere model quality.","Because ProjectedDS projects to the $\\sqrt{A}$ shell unconditionally, the 56-point swing on Can suggests the projection acts as a strong prior that can suppress useful low-norm proposals; interpolating the projection radius between 0 and $\\sqrt{A}$ would reveal whether the shell is a correction or a constraint.","The planner needs a demonstration-derived goal latent at every decision, a limitation the authors acknowledge; a learned goal encoder or task embedding would let the same steering mechanism plan toward goals that no demonstration instantiates.",""],"forward_implications":["One architecture suffices for acting and imagining: LeWAM's policy, forward, inverse and backward modes share a single latent and backbone, so no separately trained dynamics model is needed to plan.","Diffusion steering is inference-only and planner-agnostic: the same search recipe hosts CEM, MPPI and CMA-ES updates, and any flow or diffusion policy with a deterministic sampler could be steered the same way.","Projecting proposals onto the $\\sqrt{A}$ shell is load-bearing: the ablation reports closed-loop success dropping by about 5 points on Lift, 12 on Square and 56 on Can when the projection is removed.","Raw-action MPC on these human-demonstration datasets collapses every model to 3-8% closed-loop success, so the choice of decision variable, not the particular world model, is the practical failure point for WAM-based planning.","Every training mode contributes: the ablation removes forward, backward, inverse or policy training one at a time and each removal lowers latent informativeness or closed-loop success, and removing the policy mode leaves no steerable head with DS-CEM success falling to 0.8% or 0.0%.",""],"supporting_citations":[{"why":"The forward-only JEPA world model (LeWM) whose encoder-and-SIGReg latent recipe LeWAM inherits; also the key alignment baseline LeWAM must beat.","marker":"[13]"},{"why":"Supplies SIGReg, the isotropic-Gaussian regularizer that keeps the decoder-free latent from collapsing during end-to-end training from pixels.","marker":"[14]"},{"why":"Unified World Models, the multi-mode skeleton (policy, forward, inverse in one backbone) that LeWAM extends with a fourth backward mode on a decoder-free latent.","marker":"[3]"},{"why":"Cosmos Policy, the reconstruction-based world action model that RecWAM reimplements at matched scale as the main counterexample for capacity wasted on pixels.","marker":"[4]"},{"why":"Rectified flow, which defines the action head; its deterministic noise-to-action map $g(\\epsilon; z_t)$ is the object that diffusion steering searches over.","marker":"[16]"},{"why":"Golden Ticket, whose CEM-over-noise formulation and update rule diffusion steering reuses, differing only in when search happens and how candidates are scored.","marker":"[23]"},{"why":"DINO-WM, the source of goal-latent-distance planning that LeWAM's cost in Eq. (1) adopts.","marker":"[30]"},{"why":"Diffusion Policy, provider of the PushT human demonstrations and the flow-matching policy used as the acting reference and closed-loop baseline.","marker":"[25]"}],"fun_headline_variants":["Planning in policy noise space saves robot MPC from raw-action collapse","JEPA world model matches policy, plans by steering diffusion noise","Decoder-free JEPA latent reads robot state, ignores distractors, plans","Single JEPA transformer learns forward, backward, inverse dynamics, and policy"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The load-bearing premise is that the Euclidean distance between an imagined final latent and a goal latent, retrieved as the latent 25 steps after the demonstration frame nearest the current state, ranks action chunks the way real task success would; the paper does not validate this proxy against a task-reward signal.","fun_headline_variants_meta":{"raw":{"variants":["Planning in policy noise space saves robot MPC from raw-action collapse","JEPA world model matches policy, plans by steering diffusion noise","Decoder-free JEPA latent reads robot state, ignores distractors, plans","Single JEPA transformer learns forward, backward, inverse dynamics, and policy"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000185,"raw_usage":{"cost":0.000184996,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000184996,"upstream_inference_prompt_cost":9.0356e-05,"upstream_inference_completions_cost":9.464e-05},"total_tokens":1393,"prompt_tokens":1055,"completion_tokens":338,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":512,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":274}},"tokens_in":1055,"tokens_out":338,"cost_source":"openrouter_usage","duration_ms":3748,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":274,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-09T06:18:34.393902+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Score DS-CEM's proposed plans two ways on identical rollouts, by the paper's latent-distance cost $\\|\\hat{f}(z_t,a)-z_g\\|_2$ (Eq. 1) and by actual simulator success at reaching the target pose, and compare the rankings of the 192 candidates in a typical planning step; if the rankings disagree, or if noise-space planning stops beating action-space planning once a reward-based cost replaces the proxy, then the latent-distance proxy, not diffusion steering, carries the reported closed-loop gains.","supporting_citations":[{"cited_title":"LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels","cited_arxiv_id":"2603.19312","evidence_quote":"The forward-only JEPA world model (LeWM) whose encoder-and-SIGReg latent recipe LeWAM inherits; also the key alignment baseline LeWAM must beat."},{"cited_title":"LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics","cited_arxiv_id":"2511.08544","evidence_quote":"Supplies SIGReg, the isotropic-Gaussian regularizer that keeps the decoder-free latent from collapsing during end-to-end training from pixels."},{"cited_title":"Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets","cited_arxiv_id":"2504.02792","evidence_quote":"Unified World Models, the multi-mode skeleton (policy, forward, inverse in one backbone) that LeWAM extends with a fourth backward mode on a decoder-free latent."},{"cited_title":"Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning","cited_arxiv_id":"2601.16163","evidence_quote":"Cosmos Policy, the reconstruction-based world action model that RecWAM reimplements at matched scale as the main counterexample for capacity wasted on pixels."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Rectified flow, which defines the action head; its deterministic noise-to-action map $g(\\epsilon; z_t)$ is the object that diffusion steering searches over."},{"cited_title":"You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise Vector","cited_arxiv_id":"2603.15757","evidence_quote":"Golden Ticket, whose CEM-over-noise formulation and update rule diffusion steering reuses, differing only in when search happens and how candidates are scored."},{"cited_title":"DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning","cited_arxiv_id":"2411.04983","evidence_quote":"DINO-WM, the source of goal-latent-distance planning that LeWAM's cost in Eq. (1) adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Diffusion Policy, provider of the PushT human demonstrations and the flow-matching policy used as the acting reference and closed-loop baseline."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}