REVIEW 4 major objections 4 minor 32 references
World Translation claims that the hidden factors that make simulation diverge from reality can be read backward from each observed transition, and that translating these features across domains yields more accurate real-world dynamics predi
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 15:49 UTC pith:PJEZI2Y5
load-bearing objection Genuinely new idea for bypassing history-based hidden-state inference in sim-to-real dynamics, but the empirical claims need more transparency and the identifiability claim is narrower than advertised. the 4 major comments →
World Translation: Minimizing Sim-to-Real Gap with Backward Dynamics Extraction and Unpaired Domain Translation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that hidden variable h_t and domain characteristic c can be encoded into a latent dynamics feature z_t = ω(h_t, c) extracted backward from the observed transition (o_t,a_t,o_{t+1}), sufficient to reconstruct the next observation. Extraction runs from outcome to cause, so no informative history is required. Because the same hidden cause yields different transitions in each domain, a cycle-consistent map G_{S→R} translates z_t while preserving hidden-variable content. Trained on unpaired sim and real transitions and used to overwrite simulator states, the method beats history-based state-space models in high-uncertainty tasks (Go2 payload MSE 0.148 vs 0.298) and real-robot
What carries the argument
The load-bearing object is the backward-extracted dynamics feature z_t = ω(h_t, c), produced by a variational encoder that maps (o_t, a_t, o_{t+1}) to a latent distribution. The decoder reconstructs o_{t+1} from (o_t, a_t, z_t); an adversarial blind decoder keeps z_t from becoming a copy of o_{t+1}, a domain classifier forces z_t to encode domain identity, and feature-wise modulation biases the decoder to rely on z_t rather than domain cues in (o_t,a_t). The second half is a cycle-consistent translator G_{S→R} with reverse G_{R→S} acting on z_t in latent space, keeping the discriminator from separating domains using the observation context. In deployment, a simulator transition is encoded, t
Load-bearing premise
The whole method rests on the assumption that the hidden variable's effect is identifiable from a single observed transition (o_t,a_t,o_{t+1}); when two different hidden states produce the same transition, the extracted feature is ambiguous and the translation cannot transfer the true cause.
What would settle it
On a system where two distinct hidden configurations (e.g., a payload ball at rest in two different positions inside a box) produce the same observed transition, measure World Translation's single-step prediction error and a probe's R² for recovering the configuration from the transition. If error is no better than Direct Prediction's, or R² is near zero, the backward-extraction premise fails in a regime the method is designed to improve.
If this is right
- Learned dynamics models can drop the history-recoverability assumption; sudden unannounced events become learnable from their own outcome.
- Unpaired simulated and real transition data become usable for dynamics transfer, removing the requirement of matched pairs with identical hidden states.
- Policies can be trained directly on corrected simulator rollouts where every next state follows translated real dynamics, with state overwriting keeping trajectories physical.
- The advantage over history-based models should widen as tasks involve more abrupt hidden-variable events, and shrink when hidden causes leave no signature in the outcome.
- Real-robot fine-tuning on the Go2 quadruped with payload lowers angular-velocity tracking error compared with no adaptation and with domain randomization.
Where Pith is reading between the lines
- The identifiability premise is approximate even on the paper's own evidence: Table I recovers only R²=0.73–0.78 of the true force from a transition, and §VI.A concedes continuous forces leave weaker signatures; in settings with even lower R², backward extraction will approach the behavior of history-free prediction.
- Pairing the translated state target with an action-correction scheme, an extension the authors name in §VI.C, could keep the simulator's internal buffers consistent and likely extend the stable rollout horizon beyond the step-30 point where state overwriting accumulates error.
- Because the translation is unpaired, the map can be updated incrementally as new real transitions arrive, suggesting an online adaptation loop that does not require collecting matched simulation data.
- If the latent factorization into h_t and c is as clean as the classifier probes suggest, the same representation could double as a system-identification readout, estimating real friction or motor gains from the translated feature.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes World Translation, a sim-to-real dynamics-alignment method that combines backward dynamics extraction with unpaired domain translation. The central idea is to encode unobservable dynamics information (hidden variables and domain characteristics) from an already-observed transition (o_t, a_t, o_{t+1}) into a latent dynamics feature z_t, rather than inferring it from observation history as in conventional world models. A CycleGAN-style translator then maps these features between simulation and reality, and the translated feature is decoded into a real-domain next observation. The authors instantiate the framework with a VAE, a blind-decoder adversarial regularizer, a domain classifier, FiLM conditioning, and cycle-consistent translation. Experiments on G1 humanoid, Go2 quadruped, and R5 manipulator tasks compare single-step prediction error against DirectPred, RawSim, ResDyn, and RSSM, and a real-robot Go2 experiment with a payload is reported. The paper claims consistent gains, largest when unobservable factors are not recoverable from history.
Significance. The core idea is genuinely novel and potentially valuable: extracting hidden-variable information from the transition outcome instead of from history, and translating that latent representation across domains, addresses a real limitation of history-based learned dynamics models under sudden contact or payload disturbances. The paper also includes useful diagnostics (representation probes and latent-vs-observation translation ablation) and a real-robot pilot. If the empirical validation is made rigorous, the approach could be a meaningful step for learned dynamics in sim-to-real transfer. However, the current evidence is not sufficient to establish the breadth of the central claim, primarily because the identifiability assumption is only partially supported and the evaluation protocol lacks held-out data, pairing details, and statistical variance.
major comments (4)
- [§III.A, Eq. (1); Table I; §VI.A] The central claim rests on the identifiability assumption in §III.A: 'requiring only that the effect of h_t be identifiable from the observed transition.' The paper's own probe results in Table I show that the full transition (o_t, a_t, o_{t+1}) predicts the normalized external force with R²=0.73–0.78, so roughly a quarter of the hidden-variable variance is not recoverable. §VI.A concedes that continuous forces leave weaker signatures. This is not a mere caveat: if two distinct h_t produce the same observed transition, the encoder's z_t is an aliased estimate of h_t, and cycle-consistent translation will preserve that aliasing. The §I claim that backward extraction is 'more broadly reliable than history-based methods' is therefore stronger than the evidence. A direct identifiability test is needed—e.g., construct hidden-state pairs that yield identical transitions and measure whether the
- [§V-A.3, Fig. 4 (teacher-forcing protocol)] The paper does not specify how source and target transitions are paired when computing single-step error. The protocol says only that the model receives simulator transitions and error is measured against 'ground-truth' target transitions. Under the paper's unpaired-data premise (§III.C), a randomly chosen target transition does not share h_t with the simulator transition; in that case the reported MSE conflates prediction error with hidden-variable mismatch. If, instead, target transitions are selected so that h_t is shared, that pairing must be described and justified. This is essential for interpreting all quantitative comparisons (e.g., Go2 Payload 0.148 vs. RSSM 0.298).
- [§V-A.3, §V-B, §V-D; Fig. 4; Tables I–IV] Neither the simulation experiments nor the real-robot experiment report error bars or variance across seeds/trials. The real-robot result appears to be a single deployment. Without repeated trials, it is impossible to assess whether the claimed gains (e.g., R5 Force 0.401 vs. RSSM 0.515; Go2 tracking 0.1786 vs. 0.2475) are significant. The paper should provide at least 3–5 seeds for each simulation task and multiple independent trials for the robot experiment.
- [§V-A.3 and §V-B (train/eval split)] The evaluation may be circular for the target domain. The encoder, decoder, and translation models are trained on target-domain transitions sampled from D_R, and the same type of target data is used to compute the reported prediction errors. The text does not state that the evaluation transitions are held out from training, so the 'prediction error' may be reconstruction error rather than generalization error. A held-out split and a precise description of how target evaluation transitions are selected are required.
minor comments (4)
- [Fig. 2] The taxonomy box 'Absorbed inf' appears truncated; likely should read 'Absorbed in f'. Please correct.
- [§VI.C] The text refers to 'Appendix Table II' for hyperparameter transfer, but no appendix is included in the manuscript. Add the table or fix the reference.
- [Fig. 4 (left)] The diagram has duplicated 'e' labels near the target trajectory; clarify which quantity is being measured and against which prediction.
- [Eq. (8), Algorithm 1] The blind-decoder loss is described as adversarial, but the text does not fully specify gradient flow: the algorithm detaches z_t when updating ψ, yet the encoder's maximization of L_blind is not explicitly shown in Algorithm 1's update of φ, θ, ξ. Please clarify the exact stop-gradient/update recipe.
Circularity Check
No significant circularity: the derivation chain is self-contained and no prediction reduces by construction to a fitted input.
full rationale
World Translation's derivation chain is self-contained. The backward-extraction encoder is trained with a reconstruction objective on transitions (Eq. 11), and at inference it encodes simulator outcomes and translates the resulting features to the real domain; the target real transition is not fed into the encoder at prediction time. The VAE/CycleGAN training objectives are standard learning losses rather than fitted constants renamed as predictions. The paper's core assumption—'requiring only that the effect of h_t be identifiable from the observed transition' (§III.A)—is a stated condition, not a conclusion derived from itself; Table I's R²=0.73–0.78 and §VI.A's admission that continuous forces leave weaker signatures constrain the scope of the claim without making it circular. The representation probes (Tables I–II) are trained on the paper's own data and lack explicit train/test splits, which is a methodological self-referentiality concern, but it is not definitional circularity: the reported accuracies are not 1 by construction. There are no load-bearing self-citations (the reference list contains no works by the present authors), no imported uniqueness theorem, and no ansatz smuggled in via citation. The real-robot Go2 deployment provides an external benchmark. Hence the derivation does not reduce to its own inputs.
Axiom & Free-Parameter Ledger
free parameters (6)
- β (KL regularization weight) =
not reported
- λ_b (blind penalty weight) =
not reported
- σ_b (blind temperature) =
not reported
- λ_c (domain classifier weight) =
not reported
- λ_cyc (cycle consistency weight) =
not reported
- λ_id (identity loss weight) =
not reported
axioms (5)
- domain assumption The effect of hidden variable h_t is identifiable from a single transition (o_t, a_t, o_{t+1}).
- domain assumption The simulator captures the relevant physics at least approximately; phenomena entirely absent from the simulator cannot be translated.
- domain assumption Cycle consistency over unpaired dynamics features preserves hidden-variable semantics while transferring domain characteristics.
- domain assumption The decoder uses domain information from z_t rather than from (o_t, a_t) despite leakage.
- domain assumption Standard VAE and CycleGAN training assumptions hold, including balanced discriminators and stable hyperparameters across domains.
invented entities (1)
-
dynamics feature z_t
no independent evidence
read the original abstract
The gap between simulation and reality remains a fundamental challenge in deploying simulation-trained robotic policies in the real world. Real-to-sim methods narrow this gap from the real side, learning transition dynamics from real data to build a more realistic digital world. Learned dynamics models are their dominant instance. Such methods, however, face a partial observability problem: the same observation may branch to different transitions due to unobservable factors. Existing methods assume these factors can be recovered from observation history. However, this may fail whenever observation history is uninformative, such as a sudden contact event with no prior warning. To address this limitation, we propose \textit{World Translation}, which exploits a complementary strength of simulators and learned dynamics. Simulators are deterministic but physically imperfect, while learned models are accurate but underdetermined under partial observability. Rather than predicting transitions forward from history, we extract the unobservable dynamics information backward from an observed transition, then translate this feature across simulation and reality as an unpaired domain-translation problem that preserves dynamics content while transferring domain style. Experiments across humanoid, quadruped, and manipulator platforms show that our method achieves more accurate dynamics modeling than baselines, with the largest gains when unobservable factors cannot be recovered from observation history. Real-robot deployment on Go2 quadruped confirms improved policy transfer.
Figures
Reference graph
Works this paper leans on
-
[1]
Learning agile and dynamic motor skills for legged robots,
J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,”Science Robotics, vol. 4, no. 26, p. eaau5872, 2019
2019
-
[2]
Learning quadrupedal locomotion over challenging terrain,
J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,”Science robotics, vol. 5, no. 47, p. eabc5986, 2020
2020
-
[3]
Dextrah- rgb: Visuomotor policies to grasp anything with dexterous hands,
R. Singh, A. Allshire, A. Handa, N. Ratliff, and K. Van Wyk, “Dextrah- rgb: Visuomotor policies to grasp anything with dexterous hands,” arXiv preprint arXiv:2412.01791, 2024
Pith/arXiv arXiv 2024
-
[4]
Domain randomization for transferring deep neural networks from simulation to the real world,
J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in2017 IEEE/RSJ international con- ference on intelligent robots and systems (IROS). IEEE, 2017, pp. 23–30
2017
-
[5]
Sim-to- real transfer of robotic control with dynamics randomization,
X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to- real transfer of robotic control with dynamics randomization,” in2018 IEEE international conference on robotics and automation (ICRA). IEEE, 2018, pp. 3803–3810
2018
-
[6]
Solving rubik’s cube with a robot hand,
I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas,et al., “Solving rubik’s cube with a robot hand,”arXiv preprint arXiv:1910.07113, 2019
Pith/arXiv arXiv 1910
-
[7]
System identification,
L. Ljung, “System identification,” inSignal analysis and prediction. Springer, 1998, pp. 163–173
1998
-
[8]
Closing the sim-to-real loop: Adapting simula- tion randomization with real world experience,
Y . Chebotar, A. Handa, V . Makoviychuk, M. Macklin, J. Issac, N. Ratliff, and D. Fox, “Closing the sim-to-real loop: Adapting simula- tion randomization with real world experience,” in2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 8973–8979
2019
-
[9]
Analysis of randomization effects on sim2real transfer in reinforcement learning for robotic manipulation tasks,
J. Josifovski, M. Malmir, N. Klarmann, B. L. ˇZagar, N. Navarro- Guerrero, and A. Knoll, “Analysis of randomization effects on sim2real transfer in reinforcement learning for robotic manipulation tasks,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 10 193–10 200
2022
-
[10]
W. Xiao, S. Lyu, Z. Gong, R. Wang, and D. Wang, “Learning robotic policy with imagined transition: Mitigating the trade-off between robustness and optimality,”arXiv preprint arXiv:2503.10484, 2025
arXiv 2025
-
[11]
Learning latent dynamics for planning from pixels,
D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson, “Learning latent dynamics for planning from pixels,” in International conference on machine learning. PMLR, 2019, pp. 2555–2565
2019
-
[12]
Dream to control: Learning behaviors by latent imagination,
D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learning behaviors by latent imagination,” inInternational Conference on Learning Representations (ICLR), 2020
2020
-
[13]
Mastering diverse domains through world models,
D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap, “Mastering diverse domains through world models,”arXiv preprint arXiv:2301.04104, 2023
Pith/arXiv arXiv 2023
-
[14]
Td-mpc2: Scalable, robust world models for continuous control,
N. Hansen, H. Su, and X. Wang, “Td-mpc2: Scalable, robust world models for continuous control,”arXiv preprint arXiv:2310.16828, 2023
Pith/arXiv arXiv 2023
-
[15]
Sim-to- real transfer with neural-augmented robot simulation,
F. Golemo, A. A. Taiga, A. Courville, and P.-Y . Oudeyer, “Sim-to- real transfer with neural-augmented robot simulation,” inConference on Robot Learning. PMLR, 2018, pp. 817–828
2018
-
[16]
Neuralsim: Augmenting differentiable simulators with neural net- works,
E. Heiden, D. Millard, E. Coumans, Y . Sheng, and G. S. Sukhatme, “Neuralsim: Augmenting differentiable simulators with neural net- works,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 9474–9481
2021
-
[17]
Residual physics learning and system identification for sim-to-real transfer of policies on buoyancy assisted legged robots,
N. Sontakke, H. Chae, S. Lee, T. Huang, D. W. Hong, and S. Hal, “Residual physics learning and system identification for sim-to-real transfer of policies on buoyancy assisted legged robots,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 392–399
2023
-
[18]
Sim-to-real: Learning agile locomotion for quadruped robots,
J. Tan, T. Zhang, E. Coumans, A. Iscen, Y . Bai, D. Hafner, S. Bo- hez, and V . Vanhoucke, “Sim-to-real: Learning agile locomotion for quadruped robots,” inRobotics: Science and Systems (RSS), 2018
2018
-
[19]
Dropo: Sim-to-real transfer with offline domain randomization,
G. Tiboni, K. Arndt, and V . Kyrki, “Dropo: Sim-to-real transfer with offline domain randomization,”Robotics and Autonomous Systems, vol. 166, p. 104432, 2023
2023
-
[20]
Active domain randomization,
B. Mehta, M. Diaz, F. Golemo, C. J. Pal, and L. Paull, “Active domain randomization,” inConference on Robot Learning. PMLR, 2020, pp. 1162–1176
2020
-
[21]
Pilco: A model-based and data-efficient approach to policy search,
M. Deisenroth and C. E. Rasmussen, “Pilco: A model-based and data-efficient approach to policy search,” inProceedings of the 28th International Conference on machine learning (ICML-11), 2011, pp. 465–472
2011
-
[22]
Deep rein- forcement learning in a handful of trials using probabilistic dynamics models,
K. Chua, R. Calandra, R. McAllister, and S. Levine, “Deep rein- forcement learning in a handful of trials using probabilistic dynamics models,”Advances in neural information processing systems, vol. 31, 2018
2018
-
[23]
Day- dreamer: World models for physical robot learning,
P. Wu, A. Escontrela, D. Hafner, P. Abbeel, and K. Goldberg, “Day- dreamer: World models for physical robot learning,” inConference on robot learning. PMLR, 2023, pp. 2226–2240
2023
-
[24]
Sim- to-real of soft robots with learned residual physics,
J. Gao, M. Y . Michelis, A. Spielberg, and R. K. Katzschmann, “Sim- to-real of soft robots with learned residual physics,”IEEE Robotics and Automation Letters, 2024
2024
-
[25]
Grounded action transformation for robot learning in simulation,
J. Hanna and P. Stone, “Grounded action transformation for robot learning in simulation,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, 2017
2017
-
[26]
Reinforced grounded action transformation for sim-to-real transfer,
H. Karnan, S. Desai, J. P. Hanna, G. Warnell, and P. Stone, “Reinforced grounded action transformation for sim-to-real transfer,” inIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020
2020
-
[27]
Asap: Aligning simulation and real-world physics for learning agile humanoid whole-body skills,
T. He, J. Luo, W. He, T. Xiao, Y . Zhang, Z. Shi, S. Fidler, Y . Zhu, and L. Liu, “Asap: Aligning simulation and real-world physics for learning agile humanoid whole-body skills,” inRobotics: Science and Systems (RSS), 2025
2025
-
[28]
Unpaired image- to-image translation using cycle-consistent adversarial networks,
J.-Y . Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image- to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2223–2232
2017
-
[29]
Rl-cyclegan: Reinforcement learning aware simulation-to-real,
K. Rao, C. Harris, A. Irpan, S. Levine, J. Ibarz, and M. Khansari, “Rl-cyclegan: Reinforcement learning aware simulation-to-real,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 157–11 166
2020
-
[30]
Retinagan: An object-aware approach to sim-to-real transfer,
D. Ho, K. Rao, Z. Xu, E. Jang, M. Khansari, and Y . Bai, “Retinagan: An object-aware approach to sim-to-real transfer,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 10 920–10 926
2021
-
[31]
Film: Visual reasoning with a general conditioning layer,
E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville, “Film: Visual reasoning with a general conditioning layer,” inProceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018
2018
-
[32]
Umap: Uniform manifold approximation and projection for dimension reduction,
L. McInnes, J. Healy, and J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,”arXiv preprint arXiv:1802.03426, 2018
Pith/arXiv arXiv 2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.