Pith. sign in

REVIEW 4 major objections 6 minor 44 references

Towards Unraveling and Improving Generalization in World Models

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Modest latent errors can help world models generalize, and a small Jacobian penalty stabilizes the rest.

desk verdict The SDE perturbation analysis is a promising new lens on world-model errors, but Theorem 3.3 does not establish the claimed implicit-regularization effect, and the Jacobian-regularization story is asserted more than derived. read the letter →

arxiv 2501.00195 v1 pith:KA5K6DVW submitted 2024-12-31 cs.LG cs.AI

Qiaoyi Fang , Weiyu Du , Hang Wang , Junshan Zhang This is my paper · ORCID
classification cs.LGcs.AI
keywords worldmodelsreinforcementlearningimplicitregularizationstochasticdifferentialequationslatentrepresentationerrorsJacobiangeneralizationrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to explain when and why world models—latent dynamics models trained for model-based reinforcement learning—generalize to perturbed observations and unseen dynamics. Treating the latent dynamics as a continuous-time stochastic differential equation, the authors claim that small zero-drift representation errors act as implicit regularization, pushing the loss toward wider minima and improving robustness. They further claim that when representation errors have non-zero drift, the extra bias destabilizes training, and that penalizing the Jacobian norm of the latent dynamics controls this bias and reduces error propagation in long-horizon rollouts. If right, this gives a cheap, theory-grounded training modification: add a small Jacobian regularization term to the world-model loss.

What carries the argument

The central object is the coupled SDE system in Eqs. (5)-(8), with a stochastic perturbation $\varepsilon \sigma\, dt + \varepsilon \bar\sigma\, dB$ added to the latent encoder dynamics. The argument runs through a stochastic Taylor expansion of the loss around the unperturbed trajectory $x_t^0$, using the stochastic fundamental matrix $\Phi_t$ of the homogeneous SDE to express first- and second-order derivatives of the trajectory with respect to $\varepsilon$. The key identity is Theorem 3.3's expansion of $\mathbb{E} L(x^\varepsilon_t)$, whose $\varepsilon^2$ Hessian term $S$ gives the implicit-regularization effect, and Corollary 3.4's extra bias term $\tilde{R}$ for non-zero drift, whose size is bounded in terms of the Jacobian norm $\|\partial g/\partial x\|_F$.

What would settle it

Train a Dreamer-style agent on a task like Walker while explicitly injecting a controlled zero-drift Gaussian error of increasing variance into the latent encoder output, and measure test-return under perturbed observations: the theory predicts an intermediate error level outperforms both the no-error baseline and larger-error models. Finding a strictly monotone performance decline would contradict the implicit-regularization claim.

Watch

Extended reading notes

Core claim

For a world model written as a coupled SDE (encoder, sequence model, transition predictor, decoder), the paper shows that a zero-drift latent representation error of size $\varepsilon$ changes the expected loss as $\mathbb{E} L(x^\varepsilon_t) = \mathbb{E} L(x^0_t) + R + O(\varepsilon^3)$, where $R = \varepsilon P + \varepsilon^2(Q + \tfrac{1}{2}S)$. The Hessian-dependent term $S$ is non-negative for convex loss, which the authors interpret as a tendency toward wider minima and hence improved robustness. When the representation error has non-zero drift, a new bias term $\tilde{R}$ appears in the expansion, and the paper shows that this bias is controlled by the model's input-output Jacobian norm. The proposed remedy is Jacobian regularization, adding $\lambda \|J_\theta\|_F$ to the dynamics loss, which they demonstrate stabilizes training, accelerates convergence, and improves robustness to perturbed states, unseen dynamics, and injected encoder errors in MuJoCo Walker and Quadruped tasks.

Load-bearing premise

The whole analysis rests on the assumption that the discrete Dreamer/PlaNet latent dynamics can be faithfully described by the coupled continuous-time SDE system with independent Brownian motions, bounded C$^3$ coefficients, and encoder error that is a small $\varepsilon$-scaled stochastic perturbation; if the discrete latent process behaves differently from this SDE model, the expansions in Theorems 3.3 and 4.1 do not apply to real world models.

Editorial extensions

If this is right

  • If zero-drift latent representation errors act as implicit regularization, then perfectly minimizing encoder error during world-model training is not always optimal; modest errors can improve robustness to perturbed observations.
  • Because the unstable non-zero-drift bias $\tilde{R}$ is controlled by the Jacobian norm of the latent dynamics, the practical fix is simple and cheap: add a small $\lambda \|J_\theta\|_F$ penalty to the dynamics loss.
  • Jacobian regularization should reduce error accumulation over long prediction horizons, since rollout divergence is bounded by quantities exponential in the Jacobian norms $F_h$, $F_z$, $P_h$, $\bar{P}_h$ and the horizon $T$.
  • The regularization should be most valuable when latent states are corrupted by encoder errors or when tasks require longer-horizon imagination, where the baseline model's performance collapses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same SDE perturbation argument suggests a testable ordering: as encoder capacity is varied to produce very small vs. modest representation errors, there should be a non-monotone robustness curve, peaking at an intermediate error level, just as the batch-size experiment in Table 1 shows.
  • Because Theorem 3.3's regularization term is Hessian-dependent, the benefit of latent noise should be larger for loss landscapes with sharper minima; comparing world models with different initialization scales or learning-rate schedules could test this prediction.
  • The Jacobian-regularization bound on rollout divergence suggests a more direct diagnostic: measure the model's latent Jacobian norm during training and compare it against the realized error in long-horizon reconstructions; the bound predicts these track together.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies robustness and generalization of world models by modeling latent dynamics as a coupled SDE and treating latent representation errors as small stochastic perturbations. Its main theoretical claim, Theorem 3.3, is that zero-drift errors induce an explicit regularization term R = εP + ε²(Q + S/2) in the expected loss, with S nonnegative for convex losses, and that this implies a tendency toward wider minima and improved robustness. For non-zero-drift errors, Corollary 3.4 introduces an additional bias term, and the paper proposes Jacobian regularization to control it. The experimental section reports MuJoCo results on perturbed observations, unseen dynamics, exogenous encoder errors, and extended horizons, claiming that the regularizer improves robustness, stability, and convergence speed. The appendix contains proofs of the perturbation expansion, a CNN approximation theorem for latent encoders, and additional experimental details.

Significance. If the central claim were established, the paper would provide a useful theoretical explanation for an observed phenomenon in world-model RL and a computationally cheap regularizer with broad applicability. The authors should be credited for formulating the problem in an SDE framework, for explicitly analyzing error propagation in rollouts, and for running a comparatively wide experimental suite (two MuJoCo tasks, several perturbation families, encoder-error injections, and extended horizons). The appendix also contains a substantial attempt at rigorous proofs and a CNN approximation result. However, the headline theoretical conclusion is not supported by the provided expansion, and the connection between the theory and the proposed regularizer is not quantitatively established. As a result, the paper's main contribution is currently not validated.

major comments (4)
  1. [Section 3.1.1, Eq. (10)-(13)] The central claim that zero-drift latent representation errors act as implicit regularization is not established by Theorem 3.3. The expansion reads E L(xε_t)=E L(x0_t)+εP+ε²(Q+S/2)+O(ε³), and P=E[∇L(x0_t)^T Φ_t Σ_k ξ^k_t] is generically nonzero: ξ^k_t is a stochastic integral of the diffusion perturbation ¯σ^k, and there is no orthogonality between ∇L(x0_t) and Φ_t ξ^k_t. For example, the scalar case dxε_t=(1+εη)dB_t with L(x)=x²/2 satisfies Assumptions 3.1-3.2 and gives E L(xε_t)=t/2+εηt+ε²η²t/2, so P=ηt and for η<0 the expected loss decreases for sufficiently small ε. Consequently the nonnegativity of S for convex L does not imply a wider-minimum penalty or improved robustness; the signs and sizes of P and Q must be controlled before Eq. (10) can support the paper's interpretation.
  2. [Section 3.1.1, wider-minima discussion] Even if P were zero, S≥0 for convex L is not enough. Eq. (10) is an asymptotic expansion around the unperturbed trajectory x0_t, not an inequality for the training objective; Q is sign-unconstrained, the O(ε³) remainder is not quantified, and no ε-interval is given over which the ε² term dominates. The text's conclusion that 'modest latent representation errors' improve robustness is therefore not derivable from the theorem. For the nonconvex losses actually used in Dreamer-style models, the Hessian term S is not even guaranteed nonnegative. This gap is load-bearing for the paper's main message.
  3. [Section 3, Eqs. (5)-(8) and Proposition B.1] The continuous-time SDE representation of Dreamer/PlaNet is assumed rather than derived. Proposition B.1 only establishes conditional Gaussianity of z for known inputs h and s; it does not justify the coupled system with policy-dependent drift f(h_t,z_t,π(h_t,z_t)) and encoder input s_t, nor does it show that discrete Euler-Maruyama updates of the actual world model converge to system (9). Assumptions 3.1-3.2 require bounded C³ coefficients and the text asserts without proof that practical losses (reconstruction, reward, consistency) are C²; these regularity requirements are not verified for the experiments. Without this bridge, Theorem 3.3 cannot be applied to the empirical setting.
  4. [Section 4, Eq. (20) and Tables 3-4] The proposed Jacobian regularization is not shown to have the claimed theoretical effect. Equation (18) bounds the fundamental matrix Φ in terms of E sup ||∂g/∂x||²_F, but the same Jacobian norm also enters the 'beneficial' term S through Φ, so the net effect of penalizing it on R+R̃ is ambiguous; no inequality in the paper establishes that E|R̃| decreases or that the trajectory-divergence bound in Theorem 4.1 improves under the regularized loss. The empirical evidence is suggestive but not conclusive: Table 3 reports no error bars, and in the Quadruped g=6 setting the baseline point estimate (875.02) exceeds the regularized model (752.7), which is inconsistent with the claim of consistent improvement under unseen dynamics.
minor comments (6)
  1. [Eq. (13)] The stochastic integral in the definition of Q is written dB^k_t with t as both the upper limit and the integration variable; it should be dB^k_s.
  2. [Theorem 4.1 statement] The statement names Jacobian-related terms J1 and J2 and Hessian-related terms H1 and H2, but the definitions in Appendix C.1 give J0, J1, H0, and H1; the statement should be corrected.
  3. [Appendix B] The sentence 'Their distributions are not non-Gaussian in general' is self-contradictory; presumably 'are not Gaussian in general' was intended.
  4. [Footnote on page 1] The footnote reporting past NeurIPS ratings should be removed from an archival version of the paper.
  5. [Tables 3 and 4] These tables report single point estimates without standard deviations or trial counts; given the large stochasticity of MuJoCo/Dreamer training, error bars are needed to support the comparative claims.
  6. [Theorem A.6 proof] The text 'elaborated in Appendix [63]' is an unresolved citation; it should reference the relevant prior work, presumably [39].

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Theorem 3.3 is an asymptotic expansion derived from stated SDE assumptions, and the paper's conclusions do not reduce to fitted parameters or load-bearing self-citations.

full rationale

The central theoretical result, Theorem 3.3, is obtained by a standard stochastic perturbation expansion (Proposition B.5) applied to the coupled SDE system (5)-(8) under Assumptions 3.1-3.2. The terms P, Q, and S are defined as components of the Taylor expansion of the loss around the unperturbed trajectory, not as fitted parameters or as assumptions equivalent to the conclusion. The nonnegativity of S for convex losses is a mathematical consequence of the Hessian being positive semidefinite, and the interpretation connecting this term to wider minima is supported by external empirical work ([32]) and prior theoretical work ([10]), which are independent of the authors. The Jacobian regularization proposal follows from the derived bound (18) and Theorem 4.1; its weight lambda is taken from prior work ([33]) and is not fitted to the central claim. Appendix A's approximation result uses external CNN approximation theorems. There are no load-bearing self-citations, and no prediction is statistically forced by construction. The main weakness noted elsewhere — that the first-order term epsilon P in Eq. (10) can have either sign and may dominate the second-order S term — is a correctness or validity concern about whether the theorem establishes improved robustness, not a circularity of the derivation. The SDE representation itself is an explicit modeling assumption rather than a conclusion smuggled in through self-citation. Therefore the paper is not circular.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the SDE representation of latent dynamics, coefficient smoothness assumptions, a C^2 loss assumption, and a low-dimensional manifold assumption used to justify small approximation error. No new physical entity is introduced. The only tuned quantity in the empirical pipeline is the Jacobian regularization weight λ, which is chosen by hand from a previously published range.

free parameters (1)
  • Jacobian regularization weight λ = 0.01 (with 0.05 and 0.1 in the extended-horizon experiment)
    A hyperparameter chosen by hand, taken from the range recommended in [33]. The main experimental claims in Tables 2-5 are reported at this value, so the empirical conclusions depend on it.
assumptions (5)
  • domain assumption Assumptions 3.1 and 3.2: SDE drift and diffusion coefficients are bounded, Borel-measurable, C^3 with bounded Lipschitz partial derivatives, and initial values are square-integrable.
    Standard SDE well-posedness conditions, used in Theorem 3.3 and Theorem 4.1. They are stated in the paper but not verified for the neural-network coefficient functions of Dreamer-style world models.
  • domain assumption The discrete latent dynamics of world models are faithfully representable by the coupled SDE system (Eqs. 5-8) with independent Brownian motions.
    Proposition B.1 proves conditional Gaussianity only for z given known h and s; the full coupled system, including policy-dependent drift and encoder inputs, is assumed to follow the SDE model. If this fails, Theorems 3.3 and 4.1 do not apply to real world models.
  • ad hoc to paper The practical training loss L is C^2 and depends differentiably on z, h, z̃, s̃.
    Theorem 3.3 requires L ∈ C^2. The paper asserts without proof that DreamerV3 reconstruction, reward, and consistency losses satisfy this, even though DreamerV2/V3 use log-PDF terms and discrete or mixed latents where differentiability is nontrivial.
  • domain assumption Latent manifold assumption A.3-A.5: observations lie on a low-dimensional C^{k,α} manifold, the state measure is supported there with smooth density, and encoder/decoder maps are locally diffeomorphic.
    Used in Theorem A.6 to justify that CNN encoder approximation error can be made arbitrarily small, which grounds the ε-small-perturbation interpretation of latent representation errors.
  • domain assumption The policy π is a local maximizer of the value function, and Q is smooth enough for the implicit function theorem to replace π by a C^2 map ρ(h, z).
    Used in Appendix C in the proof of Theorem 4.1 to handle action dependence in the Jacobian terms. This smoothness is not verified for the learned policies used in the experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Unraveling and Improving Generalization in World Models." pith.science (2026). https://pith.science/paper/KA5K6DVW

@misc{pith2026250100195,
  author       = {Pith},
  title        = {Pith review of: Towards Unraveling and Improving Generalization in World Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KA5K6DVW}},
  note         = {Machine review of arXiv:2501.00195}
}
read the original abstract

World models have recently emerged as a promising approach to reinforcement learning (RL), achieving state-of-the-art performance across a wide range of visual control tasks. This work aims to obtain a deep understanding of the robustness and generalization capabilities of world models. Thus motivated, we develop a stochastic differential equation formulation by treating the world model learning as a stochastic dynamical system, and characterize the impact of latent representation errors on robustness and generalization, for both cases with zero-drift representation errors and with non-zero-drift representation errors. Our somewhat surprising findings, based on both theoretic and experimental studies, reveal that for the case with zero drift, modest latent representation errors can in fact function as implicit regularization and hence result in improved robustness. We further propose a Jacobian regularization scheme to mitigate the compounding error propagation effects of non-zero drift, thereby enhancing training stability and robustness. Our experimental studies corroborate that this regularization approach not only stabilizes training but also accelerates convergence and improves accuracy of long-horizon prediction.

Figures

Figures reproduced from arXiv: 2501.00195 by the authors.

Figure 1
Figure 1. Generalization against increasing degree of perturbation. [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Latent Representation Problem: The left and right denote the manifold M with lower dim dM embedded in a larger Euclidean space, with latent space Z a dM-dimensional ball in middle. Encoder and decoder as maps respectively pushing forward Q to P and P to Q. where Ai ’s are either convolutional or downsampling operators. For convolutional layers, Ai(x) = σ(Wc i x + b c i ), where Wc i ∈ R di×di−1 is a structured spars… view at source ↗
Figure 3
Figure 3. Walker. Eval (left) and train scores (right) under latent error process µt = 0, σ2 t = 5 [PITH_FULL_IMAGE:figures/full_fig_p033_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Walker. Eval (left) and train scores (right) under latent error process µt ∼ [0, 5], σ2 t ∼ [0, 5]. 33 [PITH_FULL_IMAGE:figures/full_fig_p033_4.png]
Figure 5
Figure 5. Figure 5: Quad. Eval (left) and train scores (right) under latent error process µt = 0, σ2 t = 0.1 [PITH_FULL_IMAGE:figures/full_fig_p034_5.png]
Figure 6
Figure 6. Figure 6: Quad. Eval (left) and train scores (right) under latent error process µt ∼ [0, 5], σ2 t ∼ [0, 5]. 34 [PITH_FULL_IMAGE:figures/full_fig_p034_6.png]
Figure 7
Figure 7. Figure 7: Quad. Open-loop reconstructed trajectories under zero-drift latent representation error (µt = 0, σ2 t = 5) with right and without left Jacobian regularization [PITH_FULL_IMAGE:figures/full_fig_p036_7.png]
Figure 8
Figure 8. Figure 8: Walker. Open-loop reconstructed trajectories under non-zero drift latent representation error (µt ∼ [0, 5], σ2 t ∼ [0, 5]) with lower and without upper Jacobian regularization. 36 [PITH_FULL_IMAGE:figures/full_fig_p036_8.png]
Figure 9
Figure 9. Figure 9: Extended horizon Walker task. Eval (left) and train scores (right) [PITH_FULL_IMAGE:figures/full_fig_p037_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 33 canonical work pages

  1. [1]

    Learning latent dynamics for planning from pixels

    Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. InInternational conference on machine learning, pages 2555–2565. PMLR, 2019

    2019

  2. [2]

    Dream to control: Learning behaviors by latent imagination, 2020

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination, 2020

    2020

  3. [3]

    Mastering atari with discrete world models, 2022

    Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models, 2022

    work page 2022

  4. [4]

    Mastering diverse domains through world models.arXiv preprint arXiv:2301.04104, 2023

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models.arXiv preprint arXiv:2301.04104, 2023

    arXiv 2023

  5. [5]

    Roberts, and Piotr Miłoś

    Samuel Kessler, Mateusz Ostaszewski, Michał Bortkiewicz, Mateusz Żarski, Maciej Wołczyk, Jack Parker-Holder, Stephen J. Roberts, and Piotr Miłoś. The effectiveness of world models for continual reinforcement learning.CoLLAs 2023, 2023

    work page 2023

  6. [6]

    Learning to Predict Without Looking Ahead: World Models Without Forward Prediction

    C.DanielFreeman,LukeMetz,andDavidHa.Learningtopredictwithoutlookingahead: World models without forward prediction.Thirty-third Conference on Neural Information Processing Systems (NeurIPS 2019), 2019. URLhttps://arxiv.org/abs/1910.13038

    work page Pith review arXiv 2019

  7. [7]

    Day- dreamer: World models for physical robot learning

    Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Day- dreamer: World models for physical robot learning. InProceedings of The 6th Conference on Robot Learning, volume 205 ofPMLR, pages 2226–2240, 2023

    work page 2023

  8. [8]

    Active world model learning with progress curiosity

    Kuno Kim, Megumi Sano, Julian De Freitas, Nick Haber, and Daniel Yamins. Active world model learning with progress curiosity. InProceedings of the 37th International Conference on Machine Learning (ICML), 2020

    work page 2020

Show all 44 references
  1. [9]

    Gaia-1: A generative world model for autonomous driving

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:submit/1234567, Sep 2023. Submitted on 29 Sep 2023

    2023

  2. [10]

    Noisy recurrent neural networks.Advances in Neural Information Processing Systems, 34:5124–5137, 2021

    Soon Hoe Lim, N Benjamin Erichson, Liam Hodgkinson, and Michael W Mahoney. Noisy recurrent neural networks.Advances in Neural Information Processing Systems, 34:5124–5137, 2021

    2021

  3. [11]

    Theory of deep learning iii: Explaining the non-overfitting puzzle.arXiv preprint arXiv:1801.00173, 2017

    Tomaso Poggio, Kenji Kawaguchi, Qianli Liao, Brando Miranda, Lorenzo Rosasco, Xavier Boix, Jack Hidary, and Hrushikesh Mhaskar. Theory of deep learning iii: Explaining the non-overfitting puzzle.arXiv preprint arXiv:1801.00173, 2017

    2017 arXiv

  4. [12]

    Generalization error of deep neural networks: Role of classification margin and data structure

    Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues. Generalization error of deep neural networks: Role of classification margin and data structure. In2017 International Conference on Sampling Theory and Applications (SampTA), pages 147–151. IEEE, 2017

    2017

  5. [13]

    Explicit regularisation in gaussian noise injections, 2021

    Alexander Camuto, Matthew Willetts, Umut Şimşekli, Stephen Roberts, and Chris Holmes. Explicit regularisation in gaussian noise injections, 2021

    2021

  6. [14]

    A survey of zero-shot generalisation in deep reinforcement learning.Journal of Artificial Intelligence Research, 76: 201–264, 2023

    Robert Kirk, Amy Zhang, Edward Grefenstette, and Tim Rocktäschel. A survey of zero-shot generalisation in deep reinforcement learning.Journal of Artificial Intelligence Research, 76: 201–264, 2023

    2023

  7. [15]

    Improving generalization in meta-learning via task augmentation

    HuaxiuYao,Long-KaiHuang,LinjunZhang,YingWei,LiTian,JamesZou,JunzhouHuang,and Zhenhui Li. Improving generalization in meta-learning via task augmentation. InProceedings of the 38th International Conference on Machine Learning (ICML). ICML, 2021

    2021

  8. [16]

    Improving generalization in meta-rl with imaginary tasks from latent dynamics mixture

    Suyoung Lee and Sae-Young Chung. Improving generalization in meta-rl with imaginary tasks from latent dynamics mixture. InAdvances in Neural Information Processing Systems (NeurIPS). NeurIPS, 2021. 11

    2021

  9. [17]

    Robust re- inforcement learning using offline data

    Kishan Panaganti, Zaiyan Xu, Dileep Kalathil, and Mohammad Ghavamzadeh. Robust re- inforcement learning using offline data. InAdvances in Neural Information Processing Systems (NeurIPS). NeurIPS, 2022

    2022

  10. [18]

    On the robustness of safe reinforcement learning under observational perturbations

    Zuxin Liu, Zijian Guo, Zhepeng Cen, Huan Zhang, Jie Tan, Bo Li, and Ding Zhao. On the robustness of safe reinforcement learning under observational perturbations. InInternational Conference on Learning Representations (ICLR). ICLR, 2023

    2023

  11. [19]

    The arcade learning environment: Anevaluationplatformforgeneralagents

    Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: Anevaluationplatformforgeneralagents. JournalofArtificialIntelligenceResearch , 47:253–279, 2013

    2013

  12. [20]

    Minecraft, beyond construction and survival

    Sean C Duncan. Minecraft, beyond construction and survival. 2011

    2011

  13. [21]

    A review of recurrent neural networks: Lstm cells and network architectures.Neural computation, 31(7):1235–1270, 2019

    Yong Yu, Xiaosheng Si, Changhua Hu, and Jianxun Zhang. A review of recurrent neural networks: Lstm cells and network architectures.Neural computation, 31(7):1235–1270, 2019

    2019

  14. [22]

    Tutorial on variational autoencoders.arXiv preprint arXiv:1606.05908, 2016

    Carl Doersch. Tutorial on variational autoencoders.arXiv preprint arXiv:1606.05908, 2016

    2016 arXiv

  15. [23]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  16. [24]

    World models.arXiv preprint arXiv:1803.10122, 2018

    David Ha and Jürgen Schmidhuber. World models.arXiv preprint arXiv:1803.10122, 2018

    2018 arXiv

  17. [25]

    Backpropagationappliedtohandwrittenzipcoderecognition

    Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard,andLawrenceDJackel. Backpropagationappliedtohandwrittenzipcoderecognition. Neural computation, 1(4):541–551, 1989

    1989

  18. [26]

    Approximation and optimization theory for linear continuous-time recurrent neural networks.Journal of Machine Learning Research, 23(42): 1–85, 2022

    Zhong Li, Jiequn Han, Weinan E, and Qianxiao Li. Approximation and optimization theory for linear continuous-time recurrent neural networks.Journal of Machine Learning Research, 23(42): 1–85, 2022. URLhttp://jmlr.org/papers/v23/21-0368.html

    2022

  19. [27]

    Antisymmetricrnn: Adynamicalsystem view on recurrent neural networks, 2019

    BoChang,MinminChen,EldadHaber,andEdH.Chi. Antisymmetricrnn: Adynamicalsystem view on recurrent neural networks, 2019

    2019

  20. [28]

    Symplecticrecurrentneural networks, 2020

    ZhengdaoChen, JianyuZhang, MartinArjovsky, andLéonBottou. Symplecticrecurrentneural networks, 2020

    2020

  21. [29]

    The implicit and explicit regularization effects of dropout, 2020

    Colin Wei, Sham Kakade, and Tengyu Ma. The implicit and explicit regularization effects of dropout, 2020

    2020

  22. [30]

    Michael Steele.Stochastic calculus and Financial Applications

    J. Michael Steele.Stochastic calculus and Financial Applications. Springer, 2001

    2001

  23. [31]

    Paul Louis Hennequin, R. M. Dudley, H. Kunita, and F. Ledrappier.Ecole d’ete de Probabilites de Saint-Flour XII-1982. Springer-Verlag, 1984

    1982

  24. [32]

    On large-batch training for deep learning: Generalization gap and sharp minima, 2017

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima, 2017

    2017

  25. [33]

    Roberts, and Sho Yaida

    Judy Hoffman, Daniel A. Roberts, and Sho Yaida. Robust learning with jacobian regularization, 2019

    2019

  26. [34]

    Combining pessimism with optimism for robust and efficient model-based deep reinforcement learning

    Sebastian Curi, Ilija Bogunovic, and Andreas Krause. Combining pessimism with optimism for robust and efficient model-based deep reinforcement learning. InProceedings of the 38th International Conference on Machine Learning, volume 139, pages 2210–2220. PMLR, 2021

    2021

  27. [35]

    Exploring the training robustness of distributional reinforcement learning against noisy state observations

    Ke Sun, Yingnan Zhao, Shangling Jui, and Linglong Kong. Exploring the training robustness of distributional reinforcement learning against noisy state observations. InProceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery ...

    2023

  28. [36]

    Ruida Zhou, Tao Liu, Min Cheng, Dileep Kalathil, P. R. Kumar, and Chao Tian. Natural actor- critic for robust reinforcement learning with function approximation. InAdvances in Neural Information Processing Systems. NeurIPS, 2023

    2023

  29. [37]

    Lee.Introduction to Riemannian Manifolds

    John M. Lee.Introduction to Riemannian Manifolds. Springer International Publishing, 2018. ISBN 9783319917559. doi: 10.1007/978-3-319-91755-9. URL http://dx.doi.org/10.1007/ 978-3-319-91755-9

    2018 doi

  30. [38]

    Textbooks in Mathematics

    Lawrence Craig Evans and Ronald F Gariepy.Measure theory and fine properties of functions, revised edition. Textbooks in Mathematics. Apple Academic Press, Oakville, MO, April 2015

    2015

  31. [39]

    Approximationwithcnnsinsobolev space: with applications to classification

    GuohaoShen,YulingJiao,YuanyuanLin,andJianHuang. Approximationwithcnnsinsobolev space: with applications to classification. InNeurIPS, Oct 2022

    2022

  32. [40]

    On a partial differential equation involving the jacobian determinant

    Bernard Dacorogna and Jürgen Moser. On a partial differential equation involving the jacobian determinant. Annales de l’I.H.P. Analyse non linéaire, 7(1):1–26, 1990. URLhttp://www.numdam. org/item/AIHPC_1990__7_1_1_0/

    1990

  33. [41]

    Advancedcalculus(revisededition)

    LynnHaroldLoomisandShlomoSternberg. Advancedcalculus(revisededition) . WorldScientific Publishing, Singapore, Singapore, March 2014

    2014

  34. [42]

    Differential calculus on normed spaces

    Henri Cartan. Differential calculus on normed spaces. Createspace Independent Publishing Platform, North Charleston, SC, August 2017. 13 Supplementary Materials Inthisappendix,weprovidethesupplementarymaterialssupportingthefindingsofthemainpaper on the latentrepresentation of ...

    2017

  35. [43]

    (2) rotation; and (3) noise applied to a percentage of the image,N (µ2, σ2 2). (In Walker task,µ1 = µ2 = 0.5, σ2 2 = 0.15; in Quadruped task,µ1 = 0, µ2 = 0.05, σ2 2 = 0.2.) In each case of perturbations, we examine a collection of noise levels: (1) varianceσ2 from 0.05 to 0.55...

  36. [44]

    rotation, +α◦ mask β%, N (0.5, 0.15) clean σ2 1 = 0.35 σ2 1 = 0.5 α = 20 α = 30 β = 50 β = 75 Jac Reg 967.12 742.32 618.98 423.81 226.04 725.81 685.49 Aug w.N (0.15, 0.1) 847.19 182.33 127.72 286.63 213.93 767.92 187.66 Aug w. rotation10◦ 860 286.26 184.84 695.34 424.88 347.66...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.