{"id":"9d904bbb-4f67-40a2-a9c4-fb27a93e37c5","arxiv_id":"2505.11528","paper_version":6,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Latent diffusion over DINO and SigLIP features, with iterative imagination-guided action refinement, improves robot manipulation success rates on LIBERO-LONG, CALVIN D-D, and real-world tasks.","lead":"LaDi-WM predicts future robot-object scenes in the latent space of DINO and SigLIP visual features using a diffusion model, then feeds those imagined futures back into a diffusion policy for iterative action refinement. On LIBERO-LONG, CALVIN, and a 7-task real robot, the paper reports higher success rates than behavior cloning and prior world-model baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The causal role of the world model is not isolated: the ablations never replace imagined future latents with ground-truth or non-predictive future states under the same training protocol, so the 27.9-point gain could come from extra conditioning tokens or iterative compute.","rationale":"I read the paper's central claim as an empirical causal claim: imagined future latent states produced by LaDi-WM are responsible for the reported manipulation gains. The system-level results are plausible, and the ablations (Tables 3, 9, 10) are the right kind of evidence; I credit the copy-model control and the iterative-refinement analysis for addressing simple alternative explanations. My concern is narrower: none of the ablations varies the accuracy or informativeness of the future-state input while holding architecture, conditioning tokens, and inference-time compute fixed. The copy model is a non-predictive future-state control, but it also changes the training input distribution relative to the world-model policy, so it cannot separate 'predicted future states are informative' from 'the specific extra conditioning tokens produced by this trained world model are what the policy learned to exploit.' A ground-truth oracle arm would upper-bound the value of prediction accuracy; a matched non-predictive control trained under the same protocol would lower-bound it. The paper also never reports a direct metric of future-latent prediction quality, so the sufficiency of DINO/SigLIP latents in Eq. (1) is asserted rather than demonstrated. I did not find an internal contradiction in the equations that would invalidate the method; Eq. (8) has a formatting typo but the intended losses are clear. Section 6 honestly states limitations about data scale and long-horizon compounding error, but those do not address the causal-control gap. I therefore keep the reader's CONDITIONAL verdict: the central mechanism is plausible and supported by system-level experiments, but the attribution to predictive world modeling needs one additional control before the headline gain is established.","tokens_in":14480,"tokens_out":15842,"duration_ms":170682,"concrete_test":"In LIBERO-LONG simulation, compare three policies with identical architecture, denoising steps, and two-iteration refinement, trained with (a) LaDi-WM imagined states, (b) ground-truth future latent states from expert demonstrations, and (c) a copy-of-current-latent as the 'future' state. For each policy, during evaluation use the same conditioning modality used in training; for (b), obtain future states by unrolling the simulator with the proposed action sequence. Report mean ± SD over 3 seeds. If (b) does not exceed (a), prediction quality is not the bottleneck; if (c) matches (a)'s 68.7%, the gain is not driven by predictive future information.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that LaDi-WM improves manipulation (LIBERO-LONG 40.8 to 68.7, Table 4) by refining actions with future latent states imagined by the world model. For that claim to hold, the concatenated DINO/SigLIP latent in Eq. (1) must be a sufficient and action-predictive state for robot-object dynamics, and the imagined future states must be the causal driver of the gain. The paper's ablations provide indirect support: the copy-of-current-state control (Table 9) gives no improvement, pixel diffusion is worse (Table 10), removing SigLIP or the interaction hurts (Table 3), and refinement iterations raise success (Table 4). These are meaningful. However, they do not isolate the predictive content of the imagined states from the added conditioning tokens and the extra policy passes in the iterative loop. The copy-model baseline replaces future states with the current state; it is a non-predictive control, not a control that varies prediction accuracy while holding the training input distribution and inference-time compute fixed. No experiment conditions the same policy on ground-truth future latent states, nor on an explicit non-predictive future-state baseline trained under the same protocol. Consequently, the attribution of the 27.9-point gain specifically to world-model prediction quality, rather than to the additional conditioning machinery or extra inference compute, is the least-secure link in the central argument. Direct prediction quality is never reported with concrete numbers; Fig. 3a mentions a test MSE but gives no values or units, so representation sufficiency remains an empirical premise asserted rather than tested.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"LaDi-WM proposes a latent diffusion world model that predicts future latent states in the concatenated feature space of frozen DINO and SigLIP encoders, conditioned on historical latent states and action sequences. A diffusion policy is trained to output action sequences, and at inference the policy's initial actions are fed to the world model to generate imagined future latent states, which are then used as additional conditioning for iterative action refinement. Experiments on LIBERO-LONG, CALVIN D-D, and a real 7-DOF robot setup report large gains: 27.9 percentage points over behavior cloning on LIBERO-LONG (40.8 to 68.7), improved CALVIN average sequence length (2.44 to 3.63), 15.1 points over Seer on LIBERO-LONG, and 20 points on real-world tasks. Ablations cover the diffusion module, SigLIP features, interactive cross-attention, number of refinement iterations, number of imagined frames, and diffusion space.","tokens_in":14782,"tokens_out":7992,"duration_ms":76745,"significance":"If the causal attribution holds, the paper makes a useful contribution: it shows that diffusion in a frozen VFM latent space can serve as a task-agnostic world model for manipulation and that iterative refinement with imagined latent states can substantially improve imitation learning. The paper is careful in several respects: the world model is trained on tasks distinct from policy training in both benchmarks, the ablations isolate architecture choices, the copy-model control shows that simply adding tokens is not sufficient, and the cross-scene transfer experiment (LIBERO-trained world model used for CALVIN policy) supports generalization claims. The code release is promised. The main reservation is that the core causal mechanism, predicted future states as opposed to added conditioning or iterative compute, is not isolated by the current experiments.","major_comments":[{"comment":"The central claim that the 27.9-point gain over behavior cloning on LIBERO-LONG is caused by the predictive content of the imagined future states is not isolated. The 'without imagination' row in Table 4 is a policy that outputs actions directly; moving to '1-iter imagination' changes three things at once: the policy receives additional latent tokens, an extra action-refinement pass is inserted, and the world model contributes a forward computation. The copy-model control in Table 9 is a step in the right direction, but it is not matched to the experimental protocol: the policy is trained with copied current states as input, so it can learn to ignore them, and the condition still differs from 'without imagination' in the same confounding way. No experiment holds the conditioning format and inference compute fixed while varying the predictive accuracy of the future states (e.g., an oracle condition using ground-truth future latents, a corrupted-future condition, or a fixed non-predictive feature baseline). Without such a control, the 19.9-point gain from 0 to 1 iterations and the additional 8.0 points from 1 to 2 iterations cannot be unambiguously attributed to world-model prediction quality. I recommend adding an oracle future-state condition and at least one non-predictive future-state baseline trained under the same protocol.","section":"Section 4.2, Tables 4 and 9, Appendix F"},{"comment":"The world model's state representation is the concatenation of frozen DINO and SigLIP features of the current image, and the paper never directly evaluates whether this representation is sufficient for the robot-object dynamics it must predict. The only quantitative support is downstream task success, which is exactly what the causal claim is trying to explain. No prediction error metric (e.g., latent MSE or cosine similarity between predicted and ground-truth future latents on held-out tasks) is reported; Figure 3(a) mentions test MSE only qualitatively. If the VFM latents discard task-relevant information such as object pose, contact state, or small displacements, the imagined states could not plausibly drive the reported gains, and the results would not establish the proposed mechanism. Please report direct world-model prediction quality as a function of horizon and action conditioning, and ideally ablate with oracle versus corrupted future states.","section":"Section 3.1, Eq. (1); Appendix F"}],"minor_comments":[{"comment":"The training objective has unbalanced brackets and parentheses; as printed it is not a well-formed expression. Please rewrite it with matched delimiters.","section":"Appendix A, Eq. (8)"},{"comment":"In the Seer row, the entry for Task 3 and Task 4 is printed as '48.351.7', which should be '48.3 51.7'.","section":"Table 1"},{"comment":"The text says 'our method outperforms the transformer-only architecture by 7.8%', but the stated averages are 60.7 and 52.1, a difference of 8.6 percentage points. Please correct the value or clarify the computation.","section":"Section 4.2, Table 3"},{"comment":"'World modes can interact' should read 'World models can interact'.","section":"Section 2"},{"comment":"The real-world results report only average success rates without the number of rollouts per task or any variance measure. If the 'same metric as LIBERO-LONG' implies 20 rollouts per task, state this explicitly; otherwise report the trial counts and, where feasible, confidence intervals.","section":"Section 4.3, Table 5"},{"comment":"The paper should state explicitly which equations are imported from Huang et al. [23] and what is new (the interactive cross-attention between the two latent streams), since Eqs. (3)-(6) currently appear as a restatement of the cited derivation with added subscripts.","section":"Section 3.1, Eqs. (3)-(6)"},{"comment":"The 'convergence analysis' reports an empirical entropy reduction over iterations, not a formal convergence guarantee. I suggest rephrasing 'convergence' to 'empirical stabilization' to avoid overclaiming.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The central technical gap is the missing control that isolates predictive content from extra conditioning and iterative compute. The oracle and corrupted-future experiments are feasible within the paper's scope and would substantially strengthen the causal claim. The real-world trial counts should also be reported. I do not see grounds for rejection; the architecture ablations and cross-scene transfer are solid evidence that the approach works, even if the precise mechanism is not yet fully pinned down."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper reports real gains: 27.9% over behavior cloning on LIBERO-LONG, a 3.63 average sequence length on CALVIN D-D, and 20% average success rate improvement on a real robot. The method combines latent diffusion world modeling over DINO and SigLIP features with an interactive cross-attention mechanism and iterative action refinement. That combination is genuinely new, and the empirical work is substantial for a conference paper.\n\nWhat the paper does well: the ablations are mostly coherent and isolate the main design choices. The copy-model control in Table 9 shows that just feeding extra tokens does not help; removing SigLIP, removing interaction, and removing diffusion all hurt. The iterative refinement gains are consistent and the convergence analysis in Appendix E, with variances shrinking across iterations, is a nice touch. Scaling behavior across world model data, policy data, and model size is also reported. These are meaningful pieces of evidence.\n\nThe soft spots are in proportion. The closest prior method, DINO-WM, is absent from all comparisons, which is a real gap since the paper is essentially building on that premise. The main tables lack error bars (only the appendix iteration table has them). Code is promised but not released, so independent replication is not yet possible. Table 6 has an arithmetic inconsistency in the Seer average (87.7 vs. the values that sum to about 87.2). These are fixable, but they matter.\n\nThe deeper issue is the stress-test concern: the ablations never condition the same policy on ground-truth future latent states or on a non-predictive baseline that varies prediction quality while holding everything else fixed. The copy model is a reasonable non-predictive control, but it does not answer whether the specific predictive content of the imagined states, rather than the added conditioning and iterative compute, drives the 27.9% gain. Direct prediction quality is asserted but never reported with concrete numbers or units. So the central claim is plausible but not fully nailed down.\n\nWho is this for? Researchers working on world models for manipulation, particularly those interested in leveraging visual foundation model latents. It deserves a serious referee, but the referee should push for the missing DINO-WM comparison, error bars, code, and an experiment that varies prediction quality while keeping the policy and inference compute fixed.\n\nMy recommendation: engage with it, send it to review, and expect revision.","headline":"Strong empirical results on manipulation benchmarks with a latent diffusion world model over DINO and SigLIP features, but the causal role of the imagined states is not fully isolated and the closest baseline is missing.","tokens_in":15362,"tokens_out":2325,"would_cite":true,"duration_ms":25072,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Latent-diffusion imagination lifts long-horizon robot success by 27.9 percent.","keywords":["world models","latent diffusion","robotic manipulation","visual foundation models","DINO","SigLIP","diffusion policy","iterative action refinement"],"falsifier":"A concrete falsifier: train the same policy pipeline but replace the imagined future latent states with a random latent vector, or with the current frame's latent code repeated, and measure success on LIBERO-LONG; if the gain over vanilla behavior cloning persists, the world model's future prediction is not the source of the improvement. A second check is to test a task whose success depends on sub-patch contact or object-in-hand state that DINO and SigLIP codes plausibly ignore, and see whether iterative refinement still adds the reported improvement.","tokens_in":14291,"feed_emoji":"🤖","tokens_out":5980,"duration_ms":56188,"temperature":0.7,"pith_summary":"LaDi-WM tries to show that a robot can plan better by imagining the future in the latent space of pretrained visual foundation models rather than in pixels. The paper's claim is that an interactive latent diffusion model, trained on task-agnostic video clips, can forecast how DINO geometric features and SigLIP semantic features will evolve under a candidate action sequence, and that these imagined latent states give a diffusion policy useful guidance for refining its actions. If this holds, predictive manipulation no longer needs accurate pixel-level video prediction, which has been the bottleneck for world models, and a single task-agnostic world model can transfer across scenes and tasks. The paper reports that this raises average success on LIBERO-LONG from 40.8 percent to 68.7 percent, raises CALVIN D-D average sequence length from 2.44 to 3.63, and improves real-world success by 20 absolute points.","feed_headline":"Latent-diffusion imagination lifts robot success by 27.9 percent","feed_subtitle":"Forecasting DINO and SigLIP features lets a diffusion policy refine actions and beat prior state-of-the-art.","key_machinery":"The load-bearing object is the interactive latent diffusion world model. It runs a diffusion process separately on the DINO and SigLIP latent codes, but couples the two streams by first decomposing each noisy code into a clean latent component using the analytic attenuation diffusion formulation, then feeding the clean component of one modality into the denoising network of the other via cross-attention, together with the action sequence. This lets geometric and semantic dynamics shape each other during denoising, which the paper argues aligns the two latent distributions and makes future-state prediction easier to learn. The second mechanism is the imagination-guided diffusion policy, which takes the imagined future latent states as extra conditioning and refines the action sequence in a closed loop with the world model; the paper attributes the gains to this iterative refinement, which progressively lowers the entropy of the predicted action distribution.","core_discovery":"The central discovery is that the dynamics of robot-object interaction are learnable and generalizable in the frozen latent space of visual foundation models, and that predicting those latent dynamics with diffusion beats predicting pixels. The paper concatenates DINO and SigLIP codes into a state $z_t=[f_{dino}(I_t); f_{sigl}(I_t)]$, trains an interactive latent diffusion process to sample future latent states conditioned on a history of latent states and an action sequence, and then feeds the imagined future states into a transformer-based diffusion policy that outputs actions. At inference, the policy's first action sequence is used to imagine a future, the imagined future is used to refine the action sequence, and the loop repeats until convergence, which the paper reports is reached in about two iterations. On LIBERO-LONG with ten demonstrations per task the method reports 68.7 percent average success against 40.8 percent for behavior cloning and 53.6 percent for the previous state of the art; on CALVIN D-D it reports an average sequence length of 3.63; in real-world trials it reports 60 percent versus 40 percent for vanilla behavior cloning.","pith_inferences":["Beyond the paper, the same recipe could extend to other frozen features, such as depth, object identity, or language-grounded embeddings, whenever those latent spaces are stable enough to learn dynamics over.","The method suggests a modular division of labor: one general latent-dynamics model shared across tasks, with a lightweight policy that queries it, which could enable larger shared world models trained on diverse robot data.","A natural testable extension is to use the imagined latent states as rollout sources for model-predictive control or reinforcement learning, where the low-entropy refinement behavior observed here might reduce the need for dense reward shaping.","The paper itself notes that long-horizon prediction suffers from compounding error; a testable extension would be to add memory or temporal consistency mechanisms to the latent world model and measure whether refinement gains continue beyond two iterations on longer tasks."],"forward_implications":["A world model trained on task-agnostic clips can improve a policy on tasks whose dynamics were never seen during world-model training, as on LIBERO-LONG where the world model had access only to LIBERO-90 clips.","Iterative action refinement with imagined latent states yields a monotonic gain up to two iterations and then plateaus, so the closed-loop policy-world-model loop is practical at test time without many calls.","The same latent-space formulation transfers across scenes: a LIBERO-trained world model improves CALVIN policy learning, reaching an average sequence length of 3.05 even though the scenes differ.","Scaling either world-model data or policy data improves success, and the gap over the prior state of the art persists across data scales.","A policy using a copy of the current observation as imagination gains nothing over vanilla behavior cloning, suggesting the predicted future, not the extra conditioning channel, is what produces the gains."],"supporting_citations":[{"why":"Supplies the pretrained DINOv2 geometric latent features that form one half of the state representation.","marker":"[10]"},{"why":"Supplies the pretrained SigLIP semantic latent features that form the other half.","marker":"[13]"},{"why":"Provides the analytic image attenuation formulation used to decompose clean latent components during interactive diffusion.","marker":"[23]"},{"why":"Provides the diffusion transformer architecture used as the world model's denoising network.","marker":"[24]"},{"why":"Supplies the LIBERO benchmark and demonstration data used for long-horizon evaluation and for training the world model on LIBERO-90.","marker":"[25]"},{"why":"Supplies the CALVIN benchmark used to evaluate long-horizon language-conditioned manipulation and cross-scene transfer.","marker":"[26]"},{"why":"Is the previous state-of-the-art method that LaDi-WM compares against and outperforms on both benchmarks.","marker":"[27]"},{"why":"Demonstrates the precedent of world models on pretrained DINO features, which LaDi-WM extends with semantic features and latent diffusion.","marker":"[4]"}],"fun_headline_variants":["Diffusion world model boosts robot policies by 27.9%","Robot imagination via latent diffusion lifts success 27.9%","Forecasting latent features improves robot manipulation","Latent diffusion world model enhances action refinement","Imagining futures in latent space beats pixel prediction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the concatenation of frozen DINO and SigLIP codes is a sufficient state description for predicting the future of robot-object interaction; if those features discard task-relevant detail such as contact state or small displacements, the imagined states used to refine the policy carry no useful signal.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion world model boosts robot policies by 27.9%","Robot imagination via latent diffusion lifts success 27.9%","Forecasting latent features improves robot manipulation","Latent diffusion world model enhances action refinement","Imagining futures in latent space beats pixel prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000557,"raw_usage":{"total_tokens":2674,"prompt_tokens":996,"completion_tokens":1678,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":1602}},"tokens_in":612,"tokens_out":1678,"duration_ms":11709,"temperature":1.0,"reasoning_tokens":1602,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:00:50.820627+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete falsifier: train the same policy pipeline but replace the imagined future latent states with a random latent vector, or with the current frame's latent code repeated, and measure success on LIBERO-LONG; if the gain over vanilla behavior cloning persists, the world model's future prediction is not the source of the improvement. A second check is to test a task whose success depends on sub-patch contact or object-in-hand state that DINO and SigLIP codes plausibly ignore, and see whether iterative refinement still adds the reported improvement.","supporting_citations":[{"cited_title":"Oquab, T","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained DINOv2 geometric latent features that form one half of the state representation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained SigLIP semantic latent features that form the other half."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the previous state-of-the-art method that LaDi-WM compares against and outperforms on both benchmarks."}],"review_version":1}