Pith. sign in

REVIEW 3 major objections 5 minor 19 references

Latent world models internalize only the physics their prediction objectives demand — not everything their sensors deliver

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 01:39 UTC pith:ODULFHST

load-bearing objection A genuinely controlled study showing prediction targets, not input fusion, decide which physical parameters latent world models acquire; the frontier claim survives but needs restating against the pixel-derived certificate, which the paper itself discloses. the 3 major comments →

arxiv 2607.27017 v3 pith:ODULFHST submitted 2026-07-29 cs.LG cs.RO

What Can Latent World Models Know? Physical Parameter Identifiability in Multimodal Predictive Representations

Kaizhen Tan , Xin Xu , Siru Tao , Yixiao Li , Hanzhe Hong , Yang Feng , Heqing Du , (2) Carnegie Mellon University
show 1 more author
(3) Columbia University)
This is my paper
classification cs.LG cs.RO
keywords latent world modelsphysical parameter identifiabilityjoint-embedding predictive architecturesmultimodal representation learningrecoverability certificatesconditional-mean collapsetouch sensingrobotic manipulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks which physical quantities a latent world model actually contains, and what decides that. In POKEWORLD, an interactive environment where visually identical objects hide mass, drag, and contact stiffness, the authors show that prediction targets—not inputs and not data volume—determine what enters the latent: contact stiffness is acquired only when touch is forecast (R²=0.50 vs −0.02 for fused-only touch), and a vision-only single-step model discards even perfectly visible object position. The result is an identifiability map with a frontier: drag carries a recoverability certificate of 0.89 yet plateaus near 0.13 under every deterministic prediction objective tested, while a supervised head on the same trunk reaches 0.45. The same mechanisms transfer to 4,258 real-robot episodes, where every arm missing information or prediction pressure stays flat across a fivefold data range and only the full multimodal objective forecasts force beyond a persistence baseline. If right, the work reframes world-model scaling: objective structure decides what can be learned, and data decides only how well.

Core claim

On the paper's own terms: a latent trained to predict the future carries a physical parameter only when that parameter is load-bearing for the prediction targets, and a modality fused as input but never forecast contributes nothing. Stiffness decodes at R²=0.50 only when touch is a prediction target (−0.02 when touch is merely fused in), and a vision-only single-step latent discards even visible object position (0.04). A certificate-gated protocol proves each parameter recoverable from raw observations first, so nulls are attributed to the objective; on that basis drag, certified 0.89, plateaus near 0.13 under every deterministic objective tested—the frontier. The proposed mechanism is condi

What carries the argument

The load-bearing device is a certificate-gated protocol: before any model verdict, each hidden parameter must be certified recoverable from raw observation windows by the best estimator the authors can construct (recurrent sequence probes, plus a physics-informed estimator for drag), so a null latent result can be blamed on the objective rather than the environment. On that substrate the paper trains X-JEPA, a family of action-conditioned latent world models that factorially vary which modalities enter the input (vision-only vs. vision+proprioception+touch) and which are forecast (latent embeddings only, or plus touch and/or proprioception targets), at horizons 1, 4, and 16 steps, with a swe

Load-bearing premise

The load-bearing premise is that the recoverability certificates fairly represent what the model's actual inputs afford, yet the drag certificate (0.89) comes from estimators that read the simulator's internal object state—which the pixel, proprioception, and touch streams never contain—while the pixel-derived certificate is only 0.43.

What would settle it

Run the same X-JEPA trunk on POKEWORLD with an objective whose optimum is not a conditional mean—an episode-level latent variable or belief-state objective—and probe the frozen latent for drag; the mechanism predicts readout should climb from ~0.13 toward the 0.89 certificate, and if it does not, the blind-region hypothesis fails. A companion check: recompute the drag certificate using only the model's actual input streams (pixels, proprioception, touch); a drop toward the pixel-derived 0.43 would gut the 'certified-recoverable yet unacquired' framing.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Fusing a sensor modality into a world model without forecasting it is wasted: the latent discards it, so multimodal world-model training should make every modality it wants represented a prediction target.
  • Single-step, single-modal predictive objectives can drop perfectly visible state (the lazy equilibrium); multi-horizon heads or cross-modal targets restore it, and long-horizon prediction should be asked for directly rather than composed autoregressively.
  • Scaling data cannot buy back what the objective refuses: every arm missing information or prediction pressure stayed flat across a fivefold data range, so objective and architecture choices must precede scale-up.
  • Deterministic point-prediction objectives have a certified blind region—slow, ratio-type parameters such as drag (and by extension viscosity and friction, which contact-rich robotics needs)—that supervision on the same trunk acquires (0.45 vs 0.13), naming belief states or episode-level latent variables as the next discriminating objectives.
  • The anti-collapse regularizer's strength is a precision dial: one knob controls how finely physical readouts land, and physical data of low intrinsic dimension wants roughly an order-of-magnitude lighter regularization than image-scale defaults.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Following the paper's own mechanism, the cleanest test of the blind region is an objective whose optimum is not a conditional mean—episode-level latent variables or amortized belief states—which the authors flag but do not run; POKEWORLD makes that test directly runnable, and the mechanism predicts drag should climb toward its certificate under it.
  • The frontier claim is quantitatively hostage to the certificate's observation stream: the 0.89 drag certificate comes from estimators that read the simulator's internal object state, while the pixel-derived certificate is only 0.43, so 'certified-recoverable yet unacquired' is weaker if the binding stream is what the model actually sees.
  • The attribute framework generalizes beyond simulation: real-world friction and viscosity should display the same slow-ratio signature, and a linearizing coordinate (like the log-speed channel that partially unblinded drag) is a transferable sensor-design recipe for contact-rich robotics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces POKEWORLD, a 2D interactive environment with hidden mass, drag, and contact stiffness, and proposes a certificate-gated protocol to ask which physical parameters latent JEPA world models acquire. It factorially varies input modalities, prediction targets, prediction horizons, and the SIGReg regularizer weight, and measures acquisition with linear and nonlinear probes, functional tests, closed-loop control, and a real-robot RH20T validation across two embodiments. The central claims are that inputs bound what can be known, prediction targets decide what is retained, single-step vision-only latents discard even visible object state, stiffness is acquired only under touch-as-target pressure, drag is a certified-recoverable frontier parameter that deterministic prediction objectives fail to acquire, and additional data cannot compensate for missing objective structure.

Significance. If the results hold, the paper provides one of the most controlled empirical accounts to date of physical parameter identifiability in latent world models. The factorial design, the certificate gate, the supervised system-ID head on the same trunk, the untrained-encoder passthrough bounds, the coordinate-level falsifiable prediction (log-speed unblinding), and the held-out-task splits are genuine strengths that separate this work from post-hoc probing studies. The qualitative picture—targets, not inputs, drive acquisition of contact stiffness; slow ratio-type parameters are not acquired by conditional-mean prediction objectives—is coherent and likely to influence subsequent work on world-model objectives.

major comments (3)
  1. [§4.5, App. A, App. C.2] The headline frontier claim uses a recoverability certificate computed from object-state trajectories, not from the model's actual observation stream. The abstract and Fig. 1 state that drag carries a certificate of 0.89 yet plateaus near 0.13. However, App. A states that certificate estimators read trajectories of finger state, touch, actions, and object state, and explicitly calls object state 'privileged relative to the pixel channel.' App. C.2 reports the pixel-derived recurrent drag certificate as 0.43, and a centroid-based analytic estimator as 0.05. Since X-JEPA consumes pixels, proprioception, and touch—not ground-truth object state—the relevant lower bound for drag recoverability from the binding observation stream is 0.43, not 0.89. The qualitative frontier survives (0.13 < 0.43), but the gap is materially overstated, and the phrase 'a certified-recoverable parameter acquired b
  2. [Table 1 vs. App. C.2] There is a direct inconsistency in the reported drag readout. Table 1 and the main text report model drag R² ≈ 0.13, while App. C.2 states 'Model readouts against pixel certificates: ... drag 0.30.' This is not a minor wording issue: the frontier's magnitude, the attribute-matrix fractions, and the 'plateaus near 0.13' claim all depend on which number and which evaluation condition (contact windows, glide windows, ridge vs. MLP, single-horizon vs. multi-horizon) are being quoted. The authors should specify the exact condition for every drag number and reconcile the two values, or the central quantitative claim will remain ambiguous.
  3. [§3/§4.5 and Table 1] The 'model readout / certificate' fractions are presented as if the certificate were a tight upper bound on what is acquirable from the model's inputs. But the paper itself notes certificates are lower bounds and that the state-based certificate exceeds the pixel-derived one. With a lower-bound certificate, a ratio such as 0.13/0.89 is not a well-defined 'share of recoverable information'; it is an upper bound on acquisition only under the assumption that the certificate's observation stream matches the model's. Please define the fraction explicitly with respect to the pixel-derived certificate and state the assumption clearly wherever the ratio is interpreted.
minor comments (5)
  1. [Abstract and Fig. 1] The certificate of 0.89 for drag should be qualified as 'state-trajectory certificate' and accompanied by the pixel-derived value (0.43) in the same sentence, so readers are not misled about the model's actual observation stream.
  2. [Fig. 9] The legend differentiates 'pixel-derived certificate' and 'state certificate'; please annotate which estimator and observation stream each corresponds to, and indicate the 0.43 vs. 0.89 values directly on the figure.
  3. [Table 2] The two-seed design is acceptable for screening, but for the central causal claims (e.g., target composition for position 0.17/0.09 → 0.58, stiffness 0.40 vs. -0.02) please report per-seed values and, where feasible, a small bootstrap or seed count; currently the reader cannot assess the variance of the reported R² differences.
  4. [§4.5, App. C.2] The phrase 'model readout against pixel certificates: stiffness 0.64–0.79, mass 0.43, drag 0.30' is confusing because the main-text model readouts are 0.50, 0.29, and 0.13. Clarify what is being divided by what, and why the denominator changes the number.
  5. [§5] The real-robot section appropriately limits itself to observables, but the sentence 'Objective structure decides what can be learned' in the scaling-factorial discussion reads as if physical parameters were directly measured. Consider replacing 'physical parameters' with 'task-relevant observables' in that context, consistent with the Limitations section.

Circularity Check

0 steps flagged

No significant circularity: certificates are independent lower bounds, and the frontier claim is tested by a falsifiable coordinate intervention.

full rationale

The derivation chain is self-contained and non-circular. Recoverability certificates are external supervised lower bounds fitted to ground-truth labels on raw observation windows ('A certificate is a lower bound: the best R² achieved by any estimator we construct on raw observation windows'), not quantities defined by the model's prediction objective. Model content is measured by held-out probes on frozen latents, and the causal claims are supported by factorial interventions—touch-as-input vs touch-as-target, a supervised system-ID control on the same trunk, and a pixel-reconstruction control—rather than by fitting the claimed conclusion. The disclosed certificate gate ('The certificate gate polices the matrix itself—it rejected three entangled constructions and forced the two-certificate protocol') is a transparent selection rule, not a definition of the outcome. The blind-region hypothesis is tested by a falsifiable coordinate intervention (log-speed channel raises γ's linear certificate from ≈0 to 0.52 and readout to 0.33), so the paper does not merely restate the phenomenon. The state-vs-pixel certificate discrepancy (0.89 vs 0.43) noted in App. A/C.2 is a scope/validity concern about what counts as 'raw observations' for the frontier claim; it is not circular, since the certificates are not used as prediction targets and the paper discloses the pixel-derived numbers. The Limitations section explicitly scopes claims to deterministic point-prediction objectives, again narrowing rather than presupposing the result. No load-bearing self-citation or imported uniqueness theorem appears.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The study is empirical rather than a formal derivation. Its central claims rest on the POKEWORLD simulator, the LeJEPA/LeWM recipe, and the probing protocol; the listed free parameters are hand-set knobs whose sweeps generate the dose-response results. The axioms are the modeling and measurement assumptions inherited from those choices. No new physical entities are postulated.

free parameters (5)
  • SIGReg weight lambda = 0.02 default; swept 0.005-0.3
    Hand-set regularization weight; the dose-response claim of Section 4.3 depends on sweeping it.
  • Contact-frame touch loss weight = 4 default; scaled to 0, 0.25, 1
    Hand-set loss weight in dose-response 2 (App. C.2); used to show stiffness acquisition is throttled by loss share.
  • Certificate acceptance thresholds = 0.4 / 0.4 / 0.25 for m / gamma / k
    Hand-set gate in App. A; decides which parameters pass as 'recoverable' and shapes the frontier claim.
  • Prediction horizon set = Delta in {1, 4, 16}
    Chosen by hand; central to Section 4.2's multi-horizon escape from the lazy equilibrium.
  • Glide-episode reweighting ratios = relative weights 1, 4, 16, 64
    Hand-set in dose-response 1 (App. C.2); reallocates loss mass toward the blind parameter.
axioms (5)
  • domain assumption The LeJEPA/LeWM recipe with SIGReg is an adequate substrate for a latent world model; conclusions about 'latent world models' are scoped to this recipe.
    Adopted as substrate in Sections 2 and 3; the paper studies this family, not all world models.
  • domain assumption Probing R2 on the frozen predictor state reflects latent content, with nonlinear probes and functional tests as controls.
    Standard probing assumption from Hewitt & Liang; the paper triangulates with MLP probes and behavioral tests.
  • domain assumption POKEWORLD's dynamics (semi-implicit Euler, Hertzian contacts) faithfully represent the hidden mass, drag, and stiffness parameters.
    The environment in Section 3 is simulated; the scientific claims about physical parameter acquisition depend on this fidelity.
  • standard math For squared-error point prediction, the optimal prediction is the conditional mean; this underpins the conditional-mean-collapse explanation.
    Invoked in Section 4.5; the Gaussian-NLL control leaves the mean optimum unchanged, consistent with the mechanism.
  • domain assumption RH20T at 10Hz force sampling removes stiffness-type transients, so real-robot results test mechanisms on observables rather than parameter-level identifiability.
    Stated in Section 5; the paper explicitly scopes the real-robot claims to observable mechanisms.

pith-pipeline@v1.3.0-alltime-deepseek · 19169 in / 16852 out tokens · 178534 ms · 2026-08-03T01:39:24.018253+00:00 · methodology

0 comments
read the original abstract

A central premise of latent world models is that predicting the future forces a representation to internalize the physics of its environment. Which physical quantities does a trained latent actually contain, and what decides this? We answer with controlled interventions in POKEWORLD, an interactive environment whose visually identical objects hide mass, drag, and contact stiffness. A certificate-gated protocol first certifies each parameter as recoverable from raw observations, then measures whether it enters the latent, so a null result can be attributed to the objective rather than to the environment. The resulting identifiability map has two organizing mechanisms and one frontier. Inputs limit what can be known, while prediction targets decide what is retained. Stiffness enters the latent only when touch is forecast ($R^2=0.50$, compared with $-0.02$ when the same signal is merely fused into the input), and under single-step prediction a vision-only latent discards even perfectly visible object state. Drag marks the frontier. It carries a recoverability certificate of 0.89 yet plateaus near 0.13 under every deterministic prediction objective we test, while a supervised head on the same trunk reaches 0.45. Parameters whose readout is slow and ratio-type under the sensed coordinates fall outside what these objectives acquire. On RH20T, an input-target factorial across scaling curves reproduces both mechanisms across two robots and 4,258 episodes. Every arm missing information or prediction pressure stays flat over a fivefold data range, and only the full multimodal objective forecasts force beyond a persistence baseline, with held-out gains that grow with scale. Objective structure determines which physical parameters a latent acquires, and additional data improves only the parameters it already acquires.

Figures

Figures reproduced from arXiv: 2607.27017 by (2) Carnegie Mellon University, (3) Columbia University), Hanzhe Hong, Heqing Du, Kaizhen Tan, Siru Tao, Xin Xu, Yang Feng, Yixiao Li.

Figure 1
Figure 1. Figure 1: The identifiability map. Linear-probe R2 per physical quantity as the objective is cu￾mulatively enriched (columns; POKEWORLD, contact windows), against recoverability certificates (lower bounds from raw observations; nonlinear probes agree, App. C). Multi-horizon pressure re￾covers visible state, cross-modal targets recover contact physics, and one certified-recoverable pa￾rameter (boxed) is acquired by n… view at source ↗
Figure 2
Figure 2. Figure 2: POKEWORLD and the X-JEPA factorial. Left: a force-controlled finger interacts with an object whose mass, drag, and stiffness are resampled every episode and never visible (inset: the actual 64×64 observation). Middle: each sensor stream carries different physics—sub-step tactile peaks carry stiffness, glide decay carries drag, impulse–velocity coupling carries mass. Right: the variant family factorially va… view at source ↗
Figure 3
Figure 3. Figure 3: Targets, not inputs, decide hidden-parameter content. Probe R2 by variant (gray: no touch target; orange: touch target, vision-only input; blue: full). Stiffness appears only under touch￾as-target; position emerges from target composition. Dotted: certificates. the ratio m = F/a; windowed MLPs cannot express temporally gated programs; the recurrent probe expresses both, App. A). On the model side we report… view at source ↗
Figure 4
Figure 4. Figure 4: The latent reads contact physics on held-out real episodes, on both embodiments. Top block: KUKA (53 s); bottom block: Flexiv (61 s). Each block shows camera frames at four interaction phases (native-resolution color, cropped to the manipulation zone and brightened for display—the model consumes 96×96 grayscale; orange overlay marks change since the previous panel; dotted connectors mark each frame’s time)… view at source ↗
Figure 5
Figure 5. Figure 5: The four-arm factorial across scale. Flexiv, within-embodiment (identical robot, tasks, viewpoint, protocol; fixed compute; ×: +50%-compute control). Left: position readout—the fused￾input arms sit at the passthrough bound regardless of targets (dashed: held-out tasks). Center: future￾force forecasting on held-out tasks—the passthrough-immune force metric: VFX beats persistence at every scale and improves … view at source ↗
Figure 6
Figure 6. Figure 6: The certificate-gated protocol. Every claim in the paper follows this path. Raw ob￾servations must first certify a parameter as recoverable (recoverable?); input, target, and horizon interventions then train variants on identical data; and the frozen latent is read by probes (decod￾able?) and by functional prediction and closed-loop control (functionally used?). Two testbeds instantiate it: POKEWORLD, wher… view at source ↗
Figure 7
Figure 7. Figure 7: Nothing in the rendering reveals a hidden parameter. Three matched pairs of POKE￾WORLD episodes. Within a pair the episodes start from an identical frame and are driven by an identical action sequence; only one hidden parameter differs. Filled circles mark the start (object blue, finger gold), the hollow circle the object’s final position, the blue path its trajectory (light to dark with time), and the dot… view at source ↗
Figure 8
Figure 8. Figure 8: One knob, every readout. Lightening the anti-collapse weight λ monotonically sharpens every physical probe (left) and halves long-horizon prediction error against the static baseline (right); training is stable across the full 60× range. Arrow: direction of lighter regularization [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The γ intervention ledger. Best γ readout under every intervention deployed against drag, against the pixel-derived (dashed) and state-trajectory (dotted) certificates. Prediction-side mechanisms (gray) plateau; only a linearizing sensed coordinate (orange) and direct supervision (green) move it. Dose–response 2: starving an acquired parameter. Scaling the contact-frame touch-loss weight down from its defa… view at source ↗
Figure 10
Figure 10. Figure 10: The same task on two frozen latents. One held-out episode, one planner, one budget; only the representation differs. Object path colored by time, finger path dotted, hollow circle the object’s start, shaded disc the goal region. Left: the object-blind vision-only latent (×) — the planner never reaches the object, and the distance never moves off its initial 0.55. Center: the full model (✓) maneuvers the f… view at source ↗
Figure 11
Figure 11. Figure 11: Current-force readout is passthrough-bounded. Held-out force readout of the four arms against the untrained-encoder bound (dotted): VX lacks the information, VF destroys it, VFX saturates the bound—which is why the main figure reports force forecasting instead. Cross-robot differences are a viewpoint effect. The apparent V degradation across robots is the viewpoint effect of [PITH_FULL_IMAGE:figures/full… view at source ↗

Review history (3 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 17 linked inside Pith

  1. [1]

    The SIGReg objective as variational free energy: A theoretical active-inference account of JEPA world models.arXiv preprint arXiv:2607.13612,

    Fabio Arnez and Alexandra Gomez-Villa. The SIGReg objective as variational free energy: A theoretical active-inference account of JEPA world models.arXiv preprint arXiv:2607.13612,

    Pith/arXiv arXiv

  2. [2]

    over the frame and temporal-difference channels, with small MLPs for proprioception and touch—fused and projected through a linear layer withBatchNorm. There is deliberately no trailing LayerNorm: a final LayerNorm constrains embeddings to a shell and blocks the isotropic-Gaussian target of SIGReg, reproducing the observation of Maes et al. (2026). The pr...

    2026

  3. [6]

    RH20T: A comprehensive robotic dataset for learning diverse skills in one-shot

    Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, Jirong Liu, Chenxi Wang, Junbo Wang, Haoyi Zhu, and Cewu Lu. RH20T: A comprehensive robotic dataset for learning diverse skills in one-shot. arXiv preprint arXiv:2307.00595,

    Pith/arXiv arXiv

  4. [7]

    Intuitive physics understanding emerges from self- supervised pretraining on natural videos.arXiv preprint arXiv:2502.11831,

    Quentin Garrido, Nicolas Ballas, Mahmoud Assran, Adrien Bardes, Laurent Najman, Michael Rab- bat, Emmanuel Dupoux, and Yann LeCun. Intuitive physics understanding emerges from self- supervised pretraining on natural videos.arXiv preprint arXiv:2502.11831,

    Pith/arXiv arXiv

  5. [11]

    Do generative video models understand physical principles?arXiv preprint arXiv:2501.09038,

    Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles?arXiv preprint arXiv:2501.09038,

    Pith/arXiv arXiv

  6. [12]

    Kepler- Encoder-v0.1: Towards a multimodal embedding model for robots.arXiv preprint arXiv:2607.13522,

    Ishneet Sukhvinder Singh, Dhanoosh Pooranakumaran, Alex Nguyen, and Jia Qi Yip. Kepler- Encoder-v0.1: Towards a multimodal embedding model for robots.arXiv preprint arXiv:2607.13522,

    Pith/arXiv arXiv

  7. [13]

    Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers.arXiv preprint arXiv:2409.20537,

    Lirui Wang, Xinlei Chen, Jialiang Zhao, and Kaiming He. Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers.arXiv preprint arXiv:2409.20537,

    Pith/arXiv arXiv

  8. [14]

    Wanghao Ye, Aarosh Das, Sihan Chen, Yiting Wang, Bowei Tian, Guoheng Sun, Shwai He, Zheyu Shen, Ziyao Wang, Yexiao He, Zhaoyi Liu, Meng Liu, Yuning Zhang, Meng Feng, Ziyi Wang, Yi- long Dai, Yifei Dong, Siyuan Peng, Zhenle Duan, Joshua Liu, Lang Xiong, and Ang Li. TacGen: Touch is a necessary dimension of physical-world representation – addressing tactile...

    Pith/arXiv arXiv

  9. [15]

    A control theory of predictability in latent world models.arXiv preprint arXiv:2607.10362,

    Hanzhe You, Yonggang Zhang, Maohao Ran, Zhiqin Yang, Zhenyuan Zhang, Wei Xue, Jun Song, Xinmei Tian, and Yike Guo. A control theory of predictability in latent world models.arXiv preprint arXiv:2607.10362,

    Pith/arXiv arXiv

  10. [16]

    DINO-WM: World models on pre- trained visual features enable zero-shot planning.arXiv preprint arXiv:2411.04983,

    Gaoyue Zhou, Hengkai Pan, Yann LeCun, and Lerrel Pinto. DINO-WM: World models on pre- trained visual features enable zero-shot planning.arXiv preprint arXiv:2411.04983,

    Pith/arXiv arXiv

  11. [128]

    Stiffness enters the latentonly through touch-as-target; targets compose for localization

    14 Preprint Table 2:Prediction targets decide representational content.Ridge R 2 from the frozen predictor state (contact windows; seedss 0/s1; certificates in the last row). Stiffness enters the latentonly through touch-as-target; targets compose for localization. Variant Inputs Targets logk logm Obj. pos. V vision vision−0.02/−0.02 0.15/0.14 0.04 VF all...

    2016

  12. [1999]

    LeWorld- Model: Stable end-to-end joint-embedding predictive architecture from pixels.arXiv preprint arXiv:2603.19312,

    Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, and Randall Balestriero. LeWorld- Model: Stable end-to-end joint-embedding predictive architecture from pixels.arXiv preprint arXiv:2603.19312,

    Pith/arXiv arXiv

  13. [2019]

    Sparsh: Self-supervised touch representations for vision-based tactile sensing.arXiv preprint arXiv:2410.24090,

    Carolina Higuera, Akash Sharma, Chaithanya Krishna Bodduluri, Taosha Fan, Patrick Lancaster, Mrinal Kalakrishnan, Michael Kaess, Byron Boots, Mike Lambeta, Tingfan Wu, and Mustafa Mukadam. Sparsh: Self-supervised touch representations for vision-based tactile sensing.arXiv preprint arXiv:2410.24090,

    Pith/arXiv arXiv

  14. [2020]

    Mastering diverse domains through world models.arXiv preprint arXiv:2301.04104,

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models.arXiv preprint arXiv:2301.04104,

    Pith/arXiv arXiv

  15. [2022]

    A generalization theory for JEPA-based world models.arXiv preprint arXiv:2606.27014,

    Jingyi Cui, Qi Zhang, Hongwei Wen, and Yisen Wang. A generalization theory for JEPA-based world models.arXiv preprint arXiv:2606.27014,

    Pith/arXiv arXiv

  16. [2023]

    V-JEPA 2: Self-supervised video models enable understanding, prediction and planning.arXiv preprint arXiv:2506.09985,

    Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Mojtaba, Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, Sergio Arnaud, Abha Gejji, Ada Martin, Francois Robert Hogan, Daniel Dugas, Piotr Bojanowski, Vasil Khali- dov, Patrick Labatut, Francisco Massa, Marc Szafraniec, Kapil Krishnakumar, Yong Li, X...

    Pith/arXiv arXiv

  17. [2024]

    TouchWorld: A predictive and reactive tactile foundation model for dexterous manipulation.arXiv preprint arXiv:2607.07287,

    Jianyi Zhou, Feiyang Hong, Yunhao Li, Yicheng Zhao, Yongjue Cen, Zirui Liu, Jiakang Huang, Zirui Chen, Ruiyang Zhang, Weizhuo Zhu, Xuhua Song, and Shuo Yang. TouchWorld: A predictive and reactive tactile foundation model for dexterous manipulation.arXiv preprint arXiv:2607.07287,

    Pith/arXiv arXiv

  18. [2025]

    LeJEPA: Provable and scalable self-supervised learning with- out the heuristics.arXiv preprint arXiv:2511.08544,

    Randall Balestriero and Yann LeCun. LeJEPA: Provable and scalable self-supervised learning with- out the heuristics.arXiv preprint arXiv:2511.08544,

    Pith/arXiv arXiv

  19. [2026]

    Self-supervised learning from images with a joint-embedding predictive architecture.arXiv preprint arXiv:2301.08243,

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture.arXiv preprint arXiv:2301.08243,

    Pith/arXiv arXiv