{"id":"63c1c930-bdb4-4ae9-b3db-7245866a5581","arxiv_id":"2607.24267","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Hierarchical visuo-tactile dynamics with contact-gated attention cut 10-step LPIPS to 0.058 and lift average zero-shot contact-rich planning success to 81.7%.","lead":"FeelWorld is a robot world model that predicts both future camera images and hierarchical touch states (contact, force geometry, slip), gating touch into vision only when contact is likely. On grasping and USB insertion it improves long-horizon visual forecasts and raises zero-shot CEM planning success to about 82%.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The hard contact gate is evaluated only open-loop on logged trajectories with executed actions; the planning claim rests on its accuracy under counterfactual CEM action sequences, which is never measured.","rationale":"The reader's weakest_assumption names the same load-bearing element — the hard contact gate — and my pass confirms rather than displaces it; I sharpen it from a design concern to an evaluation-gap concern: the gate's accuracy is demonstrated only on executed-action test rollouts, while the headline planning number depends on its behavior over counterfactual CEM candidates. This does not overturn the paper's evidence: the internal numbers are consistent, the ablations (Table I) isolate the gate's contribution to prediction quality, and the in-distribution onset error of 0.44 frames with ~balanced contact classes makes catastrophic gate failure unlikely on-distribution. So the appropriate posture is the reader's existing CONDITIONAL, not REJECT. The condition I'd attach is precisely the oracle-gate experiment plus a threshold/soft-gate sensitivity analysis — both cheap, both decisive. If oracle gating matches predicted gating, the central claim stands essentially as stated; if not, the planning gains are partially attributable to something other than correct contact-stage decomposition and the mechanism story weakens even if the benchmark numbers hold. The reader's other reservations (no code/data release, no CIs on 40-trial rates, fruit-grasping's 87.5% vs 70% gap being only ~7 trials and likely underpowered) remain valid secondary conditions but are not the load-bearing one.","tokens_in":15280,"tokens_out":2502,"duration_ms":90033,"concrete_test":"Oracle-gate control on the real robot: rerun the contact-aware CEM planner on all three tasks (40 trials each) with g_t taken from the DM sensor's ground-truth contact API instead of the predicted ĉ_t, keeping everything else fixed. If success stays near 81.7% (chip 82.5 / fruit 87.5 / USB 75.0), predicted-gate errors do not cascade and the concern does not land; if oracle gating materially exceeds it (e.g., USB >85%), the predicted gate is the bottleneck and the hard 0.5 switch is the weak link. Complementary offline check: re-run Table I's 10-step evaluation with the gate shifted ±1–2 frames relative to predicted onset and with a soft gate g = ĉ; a large LPIPS/FVD degradation under ±1-frame shifts (the model's own onset error is 0.44 frames) would show the fusion path is brittle to realistic onset noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader correctly located the soft spot: the binary gate g_t = I[ĉ_t ≥ 0.5] (Sec. III-C, Eq. 12; Sec. III-E, Eq. 15; Algorithm 1 lines 10–14) is a hard switch controlling both visuo-tactile fusion and the activation of tactile/slip planning costs. I want to sharpen where this assumption is least secure. Table III's contact metrics (98.6% acc, 0.44-frame onset error) are computed \"over all test frames\" — i.e., rollouts along logged test trajectories with the actions that were actually executed. But the planner evaluates 400 candidate action sequences per replanning cycle, most of which are off-distribution relative to the training data (CEM-sampled, refit to elites over 8 iterations). Contact prediction error under counterfactual actions is plausibly much larger than 0.44 frames, and it is never measured. This matters because a mistimed gate mis-scores candidates asymmetrically: if a contact-seeking sequence's predicted onset is late, its tactile-goal and slip costs activate late, so CEM systematically under-values contact-making actions (or over-values premature ones if onset is early). The 81.7% vs 49.2% planning gap is attributed to correct contact-stage decomposition, but there is no oracle-gate control showing how much of that gap survives when gating is perfect — or degrades when gating is perturbed. Secondary gaps compound this: the 0.5 threshold is never swept, no soft-gate (g = ĉ) variant is ablated (only gate on/off in Table I), and during teacher forcing the gate sees ground-truth contact (Sec. III-C), a train/test mismatch only partially covered by rollout training. Mitigating evidence exists — balanced contact classes, context noise injection, and the Fig. 6 gate trace — so this is an unmeasured risk, not a demonstrated flaw.","agreement_with_reader":"agree"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper presents FeelWorld, an action-conditioned visuo-tactile world model for contact-rich manipulation. Tactile state is decomposed hierarchically into a binary contact state, a 3D tactile latent (encoded by a frozen FG-CLTP encoder over tactile point clouds), and a slip state, all predicted by a shared latent dynamics transformer with explicit per-level supervision. A contact-gated asymmetric attention mechanism lets tactile tokens always attend to visual tokens while visual tokens attend to tactile tokens only when a hard binary gate g_t = I[ĉ_t ≥ 0.5] is open, protecting visual prediction from free-space tactile noise. Training combines teacher forcing with R-step autoregressive rollout and Gaussian context-noise injection. At inference, a contact-aware CEM planner scores candidate action sequences with a visual goal-distance term over the full horizon plus tactile-goal and slip-penalty terms that activate only after predicted contact. On chip grasping, fruit grasping, and USB insertion (Imeta-Y1 robot, 200 training / 40 test trajectories per task, 6 fps), the model improves 10-step LPIPS from 0.084 (visual-only V-JEPA 2) to 0.058, stays 61% below the visual baseline after an 80-step rollout, reaches 98.1%/83.4% F1 on contact/slip, and achieves 81.7% average zero-shot planning success vs 49.2% visual-only over 40 real-robot trials per task.","tokens_in":15685,"tokens_out":4556,"duration_ms":150014,"significance":"If the results hold, this is a solid contribution to visuo-tactile world modeling. Strengths worth naming explicitly: (i) real-robot zero-shot CEM planning evaluated over 40 trials per task on three physically distinct contact-rich tasks, including USB insertion under occlusion where the gap over the visual baseline is largest (75.0% vs 37.5%); (ii) a component-wise ablation in Table I that separates tactile input, hierarchical supervision, and gating, so the design claims are not merely asserted against external baselines; (iii) long-horizon autoregressive evaluation out to 80 steps (Fig. 5) rather than only short-horizon teacher-forced metrics; (iv) honest inclusion of ~30% failed trajectories in the evaluation set; and (v) concrete, falsifiable prediction metrics (contact/slip F1, onset error) tied to sensor-API ground truth rather than self-referential targets. The hierarchical contact/tactile-latent/slip decomposition and the gated fusion idea are simple and transferable. The main limit on significance is that the planning attribution currently rests on an untested assumption about gate behavior under counterfactual actions; if the requested controls confirm the story, the 32.","major_comments":[{"comment":"The planning attribution rests on the gate activating at the correct time under counterfactual actions, but gate accuracy is only measured open-loop. Table III (contact acc. 98.6%, onset error 0.44 frames) is computed 'over all test frames', i.e., rollouts along logged trajectories with the executed actions. The CEM planner, however, evaluates N=400 sampled action sequences per cycle (Algorithm 1, lines 3–14), most of which are off-distribution relative to the 200 training trajectories per task, and the gate g_k = I[ĉ_{t+k} ≥ 0.5] in Eq. (15) then decides when the tactile-goal and slip costs switch on. A mistimed onset mis-scores candidates asymmetrically (late onset defers tactile costs, so contact-seeking sequences are under-valued; early onset activates slip penalties on free-space motions). The 81.7% vs 49.2% gap in Table IV is attributed to correct contact-stage decomposition, but n","section":"Sec. III-E, Eq. (15), Algorithm 1; Table III; Table IV"},{"comment":"The hard threshold and binary switch are never ablated as design choices. Table I ablates only gate on/off; there is no sweep of the 0.5 threshold, and no soft-gate variant (g_k = ĉ_{t+k} in Eq. (15), or multiplicative gating by ĉ_t in Eq. (12)) is compared. A soft gate is the obvious alternative — it degrades gracefully under calibration error exactly where the hard gate is most brittle — and the manuscript's own Fig. 6(b) shows the predicted contact probability hovering near the 0.5 threshold around transitions, the regime where a hard switch is maximally sensitive to small probability errors. Given that the gate's sharpness is a central design claim ('contact-gated asymmetric attention', 'contact-aware CEM'), at least one soft-gate row in Table I and Table IV (or a threshold sensitivity curve) is needed to show the binary choice is justified rather than incidental.","section":"Sec. III-C, Eq. (12); Sec. III-E, Eq. (15); Fig. 6(b)"},{"comment":"Real-robot success rates are reported without any statistical treatment. With 40 trials per task, the binomial standard error at 75% success is ~6.8pp, so the USB result (75.0% gated vs 50.0% naive) is likely but not certainly significant, and the chip gap (82.5% vs 47.5%) is safer. Since the planning claims are the paper's headline contribution and rest on 120 total real-robot trials, Table IV should include Wilson confidence intervals or a significance test (e.g., two-proportion z-test or Fisher exact) per task. Relatedly, please state whether the same goal configurations / initial conditions were used across the three planners, and whether trial order was randomized, to rule out systematic drift effects (object wear, sensor drift) confounding the comparison.","section":"Table IV, Sec. IV-D"},{"comment":"Train/inference mismatch of the fusion pathway is only partially stress-tested. During teacher forcing, g_t uses ground-truth contact labels (Sec. III-C: 'subsequent gates also use ground-truth contact labels'), so the visual pathway is trained under clean gating; the predicted gate is exercised only during R=4-step rollout training and inference. The 10-step and 80-step evaluations (Table I, Fig. 5) do use the predicted gate, which mitigates this concern for prediction quality, but the CEM planning horizon H_p=6 with replanning every 2 steps means the gate under its own predictions is the operating regime that matters most — and it is exactly the regime covered only indirectly. Please clarify explicitly in Sec. III-C/III-D whether gradient flows through g_t when it is derived from the predicted ĉ_t during rollout training (the indicator is non-differentiable; presumably the gate is deta","section":"Sec. III-C vs. Sec. III-D, Eq. (12)-(14)"}],"minor_comments":[{"comment":"Fig. 2 contains leftover placeholder text '在此处键入公式' ('type formula here') inside the planning-cost box. This must be removed; it also suggests the figure should be proofread for other rendering artifacts.","section":"Fig. 2"},{"comment":"h_goal in Eq. (15) is never defined operationally. Since tactile goals are 'physically unreachable before contact', please state how h_goal is obtained (a goal tactile observation from a reference successful trajectory? the tactile reading at the goal image's timestep?) and how D_τ behaves when the goal tactile latent was recorded on a different object instance.","section":"Sec. III-E, Eq. (15)"},{"comment":"Table III: the slip offset error is reported as '–' without explanation (contact offset is 0.69 frames). Either report it or state why it is undefined/not meaningful for slip.","section":"Table III"},{"comment":"The onset/offset errors are in frames at 6 fps (0.44 frames ≈ 73 ms). Since contact onset is sub-frame at the 30 Hz recording rate, please clarify how ground-truth onset times were localized (sensor API timestamps at 30 Hz mapped to 6 fps frames?) and whether the 0.44-frame figure is limited by label quantization rather than model precision.","section":"Table III, Sec. IV-A.1"},{"comment":"Table I: the 'FeelWorld (w/o hierarchical)' row shows Chamfer 0.072 but '–' for both F1 columns. Please state precisely which heads/losses this ablation removes (contact and slip heads only, keeping L_tis?), since 'hierarchical' is used loosely in the text.","section":"Table I"},{"comment":"Fig. 5 caption should state explicitly that rollouts are open-loop along the executed actions of test trajectories; readers may otherwise assume counterfactual rollouts. The same clarification applies to Table I.","section":"Fig. 5"},{"comment":"Reproducibility: the paper would be strengthened by a statement on code/data release. The dataset (200+40 trajectories × 3 tasks on a specific Imeta-Y1/DM-Tac W2 platform) is not publicly benchmarked, so release matters more than usual for follow-up comparison.","section":"Sec. IV-A.1"},{"comment":"Loss weights λ_c = λ_s = 0.03 vs λ_τ = 0.3 differ by an order of magnitude; a one-line justification (gradient magnitudes? a small sweep?) would help. Similarly σ = 0.007 for context noise injection (Eq. 13) is asserted to 'approximate the prediction error distribution' — a brief empirical check would support this.","section":"Eq. (10), Eq. (13), Sec. IV-A.2"},{"comment":"'Zero-shot planning' (Sec. IV-D) is used in the DINO-WM sense (no task-specific policy training); a short clarifying phrase would prevent misreading, since the world model itself is trained per-task on task data.","section":"Sec. IV-D"}],"recommendation":"major_revision","confidential_remarks":"Two observations for the editor only. (1) The pipeline depends on the authors' own prior encoders (CLTP [25], FG-CLTP [26]) as frozen backbones; this is a legitimate dependency rather than circular evaluation, but it does mean the tactile representation quality is not independently validated against alternatives (e.g., Sparsh [19]), and the paper's gains are conditional on that encoder. (2) The placeholder text left in Fig. 2 and the dense cluster of 2026 arXiv citations suggest the manuscript was prepared quickly in a fast-moving area; neither is disqualifying, but a careful revision pass is warranted. The requested oracle-gate control is the item I would treat as a condition of acceptance; it is cheap to run on existing logged data and directly tests the paper's central attribution."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a competent contact-rich world-model systems paper. The package that matters is explicit three-level tactile supervision (contact → 3D latent → slip), contact-gated asymmetric attention that keeps a visual-only path in free space, and a CEM cost that only turns on tactile/slip terms after predicted contact. Against their own V-JEPA 2 baseline they get real numbers: 10-step LPIPS 0.084→0.058, ~61% lower LPIPS at 80-step AR rollout, contact/slip F1 98.1%/83.4%, and average zero-shot planning ~81.7% vs ~49% vision-only across chip, fruit, and USB.\n\nWhat is actually new is compositional, not paradigm-shifting. VT-WM, ContactWorld, Dream-Tac, VT-WAM, Tactile-WAM, OmniVTA already put touch in world/action models. FeelWorld’s distinctive bit is the hierarchical readout plus the hard gate on both fusion and planning cost, with ablations that separate hierarchical heads from the gate (Table I) and naive joint CEM from contact-aware CEM (Table IV). Phase-wise contact-transition LPIPS (Table II) and long-horizon curves (Fig. 5) are the right experiments. Losses supervise frozen encoder targets and sensor API labels; planning success is on the robot. That is clean empirical structure, not circular.\n\nSoft spot, in proportion: the binary gate g=I[ĉ≥0.5] is load-bearing for attention and for Algorithm 1, and contact metrics are on logged test trajectories with executed actions. CEM scores hundreds of off-policy candidates per cycle; contact error under those counterfactuals is never measured, and there is no oracle-gate or soft-gate (g=ĉ) planning control. Threshold 0.5 is not swept. Teacher forcing uses GT contact; rollout+noise only partly closes that gap. That is an unmeasured risk for how much of the 81.7% gap is “correct stage decomposition” versus lucky gate behavior—not a demonstrated collapse. Secondary: no code/data, no CIs on 40-trial rates, FG-CLTP and DM-Tac stack, free λs and CEM weights.\n\nWho it is for: people building action-conditioned models for contact-rich manip who need a concrete fusion/planning recipe. Math is standard L1/BCE/focal + transformer dynamics; citations cover the 2025–26 cluster honestly. I would send it to peer review. Engage if you work in this lane; skim the gate/planning sections if you only need the idea.","headline":"Solid systems paper: hierarchical touch + contact-gated fusion actually moves LPIPS and real-robot CEM success; the hard gate under counterfactual actions is the unmeasured risk, not a shown failure.","tokens_in":16654,"tokens_out":657,"would_cite":true,"duration_ms":16733,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A world model that jointly imagines vision, contact, force geometry, and slip plans contact-rich robot actions more accurately than vision alone.","keywords":["visuo-tactile world model","hierarchical contact prediction","contact-gated attention","slip prediction","CEM planning","contact-rich manipulation","latent dynamics","3D tactile point clouds"],"falsifier":"Run the same chip, fruit, and USB tasks with systematically delayed or advanced contact labels (or sensors that chatter near onset) and check whether 10-step and 80-step LPIPS, contact/slip F1, and CEM success collapse once the gate is wrong at transition frames.","tokens_in":16229,"feed_emoji":"🤖","tokens_out":963,"duration_ms":24820,"temperature":0.7,"pith_summary":"Robots that plan by imagining futures usually only simulate how things look, so their rollouts can look right while violating contact physics. FeelWorld is a shared latent dynamics model that also predicts three hierarchical tactile states—whether contact has occurred, a 3D tactile latent encoding force-related deformation, and whether slip is starting—and uses those states both to ground the imagined future and to score candidate actions. A contact gate keeps free-space tactile noise from corrupting the visual stream and only turns on joint visuo-tactile fusion and tactile planning costs once contact is predicted. On chip grasping, fruit grasping, and USB insertion, this cuts short-horizon visual error, keeps long rollouts far more stable, and raises zero-shot model-based planning success to about 82 percent on average. The practical claim is that structured, selectively fused touch is what makes imagined contact futures physically usable.","feed_headline":"Touch-aware world model lifts contact planning to 82%","feed_subtitle":"Gating force and slip into imagined futures cuts visual drift and roughly doubles grasp and insertion success.","key_machinery":"Contact-gated asymmetric attention: tactile tokens always ground themselves in vision, but visual tokens query tactile features only when a predicted binary contact gate is open, so free-space sensor noise cannot degrade appearance dynamics while contact-phase dynamics become jointly visuo-tactile.","core_discovery":"FeelWorld shows that jointly predicting future visual latents together with hierarchical tactile states—contact, a force-related 3D tactile latent, and slip—under a contact-gated asymmetric attention mechanism yields imagined futures that are both visually sharper and physically more consistent than vision-only or naive visuo-tactile baselines, and that those predicted contact and slip states enable contact-aware CEM planning with substantially higher zero-shot success on contact-rich tasks.","pith_inferences":["The same contact gate could be reused as a curriculum signal for policy learning: train approach with vision only, then unlock tactile losses only after the model’s own contact head fires.","If slip F1 remains the weakest link under class imbalance, denser temporal tactile context or multi-finger slip heads may be the next bottleneck for insertion under occlusion.","Hard binary gating may need a soft or uncertainty-aware version for deformable or multi-contact scenes where “contact” is graded rather than instantaneous.","Pairing the world model with a fast policy evaluator, as the authors sketch, would test whether hierarchical tactile rollouts still help when CEM’s sample budget is no longer affordable."],"forward_implications":["World models for contact-rich work should predict contact, local force geometry, and slip as separate supervised levels rather than one undifferentiated tactile target.","Free-space tactile readings should be gated out of visual dynamics and planning costs until contact is predicted.","Contact-aware CEM can switch from vision-only approach costs to joint visuo-tactile goals after predicted contact, avoiding unreachable pre-contact tactile objectives.","Autoregressive rollout training with context noise reduces compounding error enough for multi-second imagined horizons to stay usable for planning.","Zero-shot planning success on fragile grasp and occluded insertion can roughly double relative to a strong visual-only latent world model when hierarchical touch is added this way."],"fun_headline_variants":["FeelWorld jointly predicts vision and touch for 82% planning success","Contact-gated world model cuts visual drift, hits 81.7% zero-shot success","Hierarchical tactile states sharpen imagined futures in contact tasks","Visuo-tactile model lifts grasp and insertion via force and slip states","Gated touch prediction keeps 80-step rollouts 61% sharper than vision-only"],"cache_read_input_tokens":128,"weakest_assumption_plain":"A hard on/off contact threshold is enough to decide when touch is pure noise versus when it should fully reshape both the visual prediction and the planner’s cost—so gating mistakes do not cascade into bad fusion or unreachable objectives.","fun_headline_variants_meta":{"raw":{"variants":["FeelWorld jointly predicts vision and touch for 82% planning success","Contact-gated world model cuts visual drift, hits 81.7% zero-shot success","Hierarchical tactile states sharpen imagined futures in contact tasks","Visuo-tactile model lifts grasp and insertion via force and slip states","Gated touch prediction keeps 80-step rollouts 61% sharper than vision-only"]},"model":"grok-4.5","effort":"low","cost_usd":0.003652,"raw_usage":{"total_tokens":1199,"prompt_tokens":831,"num_sources_used":0,"completion_tokens":84,"cost_in_usd_ticks":36524000,"prompt_tokens_details":{"text_tokens":831,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":284,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":831,"tokens_out":84,"duration_ms":6641,"temperature":1.0,"reasoning_tokens":284,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T19:37:08.169367+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the same chip, fruit, and USB tasks with systematically delayed or advanced contact labels (or sensors that chatter near onset) and check whether 10-step and 80-step LPIPS, contact/slip F1, and CEM success collapse once the gate is wrong at transition frames.","supporting_citations":[],"review_version":1}