Pith. sign in

REVIEW 6 major objections 5 minor 40 references

DoublyAware: Dual Planning and Policy Awareness for Temporal Difference Learning in Humanoid Locomotion

T0 review · 6 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DoublyAware speeds humanoid locomotion learning by splitting uncertainty into planning and policy parts.

desk verdict Solid empirical direction undermined by an invalid conformal coverage claim; needs substantial correction before acceptance. read the letter →

arxiv 2506.12095 v1 pith:VMFF3LH6 submitted 2025-06-12 cs.RO

Khang Nguyen , An T. Le , Jan Peters , Minh Nhat Vu This is my paper
classification cs.RO
keywords humanoidlocomotiontemporaldifferencemodelpredictivecontrolconformalpredictionuncertaintydecompositiongroup-relativepolicyoptimizationsampleefficiencyreinforcementlearningBench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a major obstacle to sample-efficient humanoid locomotion in model-based RL is the conflation of two distinct uncertainties: the environment's random variability in planning and the policy's incomplete knowledge during learning. To address this, DoublyAware splits these components and handles each separately: conformal prediction filters candidate trajectories from the MPPI planner using quantile-calibrated risk bounds, while a group-relative policy constraint (GRPC) regularizes policy updates in latent action space with an adaptive trust region. On the HumanoidBench suite with the Unitree H1-2 humanoid, the method reports faster convergence and more kinodynamically feasible motions than SAC, BC-SAC, AWAC, TD-MPC2, and TD-M(PC)2 on six of nine locomotion tasks. The intended upshot is that structured uncertainty modeling makes TD-MPC-style learning more data-efficient and reliable for whole-body control.

What carries the argument

The central machinery is a two-part uncertainty treatment. First, a conformal quantile filter: at each planning step, candidate trajectories from the policy network and the MPPI sampler are scored by their normalized TD value estimate, a quantile threshold (Eq. 6) is computed from the union's nonconformity scores, and only trajectories within the $(1-\alpha)$ prediction set are used to update the sampling distribution. Second, the Group-Relative Policy Constraint (GRPC): a softmax-weighted group advantage (Eq. 9) replaces absolute advantage estimates in the policy objective, and a KL divergence penalty to the MPPI-derived behavior prior (Eq. 13) enforces an adaptive trust region. Together they are meant to keep planning robust to aleatoric noise and learning stable under epistemic uncertainty.

What would settle it

Run the planner on a fixed hard task, record the empirical coverage of the conformal set (the fraction of retained high-return trajectories) as the value function is updated; if during early training the coverage falls below $1-\alpha$ for a sustained period, or if artificially freezing the value function changes the success rate significantly, the weak-teacher or exchangeability assumption is violated.

Watch

Extended reading notes

Core claim

DoublyAware is an uncertainty-aware extension of TD-MPC that explicitly decomposes overall uncertainty into planning uncertainty (aleatoric, from stochastic dynamics and sampling-based optimization) and policy uncertainty (epistemic, from incomplete policy knowledge). For planning, it builds a conformal prediction set over the union of policy-rollout and MPPI candidate trajectories, computes nonconformity scores from the TD value estimate, and retains only trajectories below a quantile threshold, which guarantees marginal coverage under exchangeability and asymptotic weak validity under a sub-linear teaching schedule. For learning, it replaces standard advantage normalization with a group-relative softmax advantage (GRPO-style) and adds a KL trust-region penalty to the policy prior, reducing gradient variance and keeping updates close to the behavior policy. Evaluated on HumanoidBench with the Unitree 26-DoF H1-2, DoublyAware achieves faster convergence and better motion feasibility than the compared RL baselines on standing, walking, running, sitting, pole navigation, hurdling, and slide-walking tasks, while remaining unsolved on crawling and stair-climbing.

Load-bearing premise

The load-bearing premise is that the policy rollouts and MPPI samples are statistically interchangeable, and that the learning value function can serve as a weak teacher whose teaching schedule satisfies the sub-linearity condition; without this, the conformal filter's coverage guarantee breaks.

Editorial extensions

If this is right

  • If a model-based RL agent separates aleatoric planning noise from epistemic policy uncertainty, it can explore in a targeted way without sacrificing statistical guarantees.
  • Conformal filtering can be applied to latent trajectory selection inside TD-MPC without modifying the underlying planner, giving distribution-free coverage even with an imperfect value function as long as a mild schedule condition holds.
  • Replacing normalized advantages with softmax group advantages reduces policy-gradient variance, leading to more stable updates and faster convergence in high-dimensional continuous control.
  • The method's success on six of nine HumanoidBench tasks suggests that structured uncertainty modeling is a practical lever for data-efficient humanoid locomotion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the same disjoint uncertainty decomposition could be applied to other MBRL planners (e.g., CEM or iCEM) beyond MPPI, since the conformal filter is model-agnostic; this is not tested in the paper.
  • The weak-teacher schedule assumption suggests the benefit of conformal filtering should grow as the value function improves; a testable prediction is that early in training the filter is loose and becomes tighter as learning progresses, and coverage may degrade if the value function is updated too aggressively.
  • The failure on crawling and stair-climbing may indicate that the current conformal score based on the TD value estimate is not sensitive to whole-body coordination constraints; replacing the score with a kinematic-feasibility-aware cost could extend the method.
  • The KL trust region on the policy prior might be adapted with automatic coefficient tuning, since GRPC's beta is fixed; adaptive beta could improve performance on tasks that need more exploration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. DoublyAware proposes an uncertainty-aware extension of TD-MPC for humanoid locomotion. It decomposes uncertainty into planning (aleatoric) and policy (epistemic) components. For planning, it uses conformal prediction to filter a union of policy rollouts and MPPI candidate trajectories based on normalized nonconformity scores from the TD value estimate (Eq. 4), claiming finite-sample marginal coverage (Eq. 5) and asymptotic weak validity (Eq. 8). For learning, it introduces GRPC, a group-relative policy constraint derived from GRPO with softmax advantage scores and a trust-region penalty. Experiments on nine HumanoidBench tasks with the H1-2 humanoid compare DoublyAware against SAC, BC-SAC, AWAC, TD-MPC2, and TD-M(PC)2, reporting faster convergence and qualitatively more feasible motion on six of nine tasks, with ablations isolating the CP and GRPC contributions.

Significance. If the theoretical claims were valid, the work would offer a principled way to handle aleatoric uncertainty in TD-MPC planning and to regularize policy learning, with a plausible empirical payoff on a relevant humanoid benchmark. The paper includes useful ablations and qualitative behavior comparisons, and it targets an important problem. However, as written, the conformal validity and asymptotic weak-validity results are not established: Eq. 5 is essentially a tautological statement about an empirical quantile of the same scores used for selection, the exchangeability assumption is violated by the online MPPI feedback, and Eq. 8 is asserted without proof or construction. The GRPC variance-reduction argument is also under-derived, and the trust-region objective is not specified precisely. The contribution therefore currently reduces to a heuristic extension with suggestive but statistically unquantified empirical evidence.

major comments (6)
  1. [Section III-B, Eqs. (4)-(7)] Equation (5) does not follow from split-conformal theory. The same nonconformity scores are used both to compute the quantile qhat in Eq. (6) and to define the selected set A_conformal in Eq. (7); there is no calibration/test split, and no independent future trajectory is scored against a quantile learned from separate data. The statement that Vtheta(z,a) belongs to A_conformal with probability at least 1-alpha is therefore not a finite-sample marginal coverage guarantee but a property of the empirical quantile of the very same scores. A valid conformal argument would require either a disjoint calibration set or a leave-one-out construction, neither of which appears in Alg. 1.
  2. [Section III-B and Alg. 1, lines 15-17] The exchangeability assumption underlying Eq. (5) is asserted rather than shown and is violated by the algorithm's own feedback. The set A_pi is sampled from the policy network and A_mppi from the MPPI sampling distribution; these are different samplers, so their scores need not be exchangeable even at a single planning step. More importantly, Alg. 1 updates the MPPI sampling distribution (mu_j, sigma_j) using A_conformal (lines 15-17), so the scores at the next planning step depend on the previously accepted set. This destroys the exchangeability between calibration scores and candidate scores that Eq. (5) requires. The paper needs either an explicit online-conformal argument that accounts for this feedback or a clear statement that Eq. (5) is only a heuristic motivation.
  3. [Section III-B, Eq. (8)] The asymptotic weak-validity claim is unsupported. The teaching schedule L = {n_k}_{k>=1} and the sub-linearity condition lim_{k->infinity} n_k/n_{k-1} = 1 are never connected to the algorithm: n_k is not defined, the schedule is never constructed, and no theorem or proof is given for Eq. (8). Without a precise online-conformal result, this equation should be removed or replaced by a rigorously stated theorem with assumptions that are checked for Alg. 1.
  4. [Section III-C, Eqs. (10)-(11)] The variance-reduction argument for GRPC is not a derivation. The losses L_softmax and L_std-norm are never defined, the variance operators in Eq. (10) are ambiguous (variance over what randomness: the sampled actions, the Q estimates, or the gradient estimates?), and Eq. (11) is a statement about norms, not variances. The conclusion that softmax-based advantages give more stable policy updates therefore does not follow from the stated assumptions. The authors should either provide a self-contained derivation or explicitly label this as a heuristic.
  5. [Section III-C, Eq. (13) and Alg. 2, line 22] The trust-region objective is not implemented as described. Eq. (13) adds beta log mu(a|s) to the group-relative term, but a KL trust-region penalty would involve an expectation over the policy ratio or an explicit KL divergence; L_KL is not defined in Eq. (13) or in Alg. 2, and Alg. 2 line 22 refers to beta L_KL without specifying L_KL. Moreover, the sentence 'mu_k denotes the behavior policy at kth iteration from D in Eq. 3' is inconsistent with Eq. (3), which defines the MPPI mean and variance update. The algorithm as written cannot be reproduced from the paper.
  6. [Section IV-A, Fig. 4 and Table I] The empirical comparison is difficult to evaluate because no number of seeds, error bars, or standard deviations are reported for the learning curves in Fig. 4, and Table I uses subjective symbols (check, circle, cross) without a scoring rubric. Since the main quantitative claim is improved sample efficiency and convergence, the authors should report multiple seeds and provide a quantitative success criterion for Table I.
minor comments (5)
  1. [Alg. 1, line 18] The returned action samples from N(mu, sigma^2 I) but mu and sigma are not defined; presumably mu_J and sigma_J are intended.
  2. [Section III-B, after Eq. (7)] The text says 'A_conformal contains the best trajectories with probability at least 1-alpha', which conflates 'best' with 'conforming'; this is an overclaim given the issues with Eq. (5).
  3. [Section III-C, Eq. (9)] The symbol tau is used for the temperature in Eq. (9) and also for the MPPI temperature in Eq. (3); please use distinct symbols to avoid ambiguity.
  4. [References] Several references are incomplete or informal, such as [9] without a year or venue details and [20] without proceedings information; these should be completed.
  5. [Introduction] The phrase 'we proposed DoublyAware' should be 'we propose DoublyAware'; minor copyedit.

Circularity Check

1 steps flagged · score 6.0 of 10

Eq. 5's conformal coverage guarantee is self-definitional: Aconformal is defined as the empirical 1−α quantile of the same scores used to build qhat, so 'coverage' is a tautology rather than split-conformal validity.

  1. self definitional [Section III-B (Eqs. 4-7) and Alg. 1 lines 13-15]
    "ˆq = quantile({¯s(i)} Nπ+N i=1, (1−α)) ... Aconformal = {a(j) 1:H ∈ A | ¯s(j) ≤ ˆq} ... 1 − α + 1/(Nπ+N)+1 ≥ P[Vθ(z, a) ∈ Aconformal] ≥ 1 − α."

    The conformal prediction set Aconformal is defined as the subset of trajectories whose nonconformity scores are at or below the empirical (1−α) quantile qhat, and qhat is computed from exactly the same scores. Standard split-conformal validity requires a separate calibration set and a new exchangeable test score; here the 'test' trajectories are the same data used to form the quantile. Consequently, Eq. 5 is a restatement of the definition of the empirical quantile (the accepted set contains about 1−α of the available scores by construction), not finite-sample marginal coverage for a future trajectory. The claimed distribution-free validity therefore reduces, by construction, to the quantile definition.

full rationale

The only place where the paper's formal claims reduce to their inputs is the conformal coverage argument. The HumanoidBench comparisons, ablations, and qualitative results are independent empirical evaluations and show no circularity. Self-citations (e.g., Peters et al. [16]) appear as background, not as load-bearing evidence. The conformal step is different: qhat in Eq. 6 is built from all scores, and Aconformal in Eq. 7 is the level set of those same scores, so the claimed coverage bound in Eq. 5 is tautological at the level of the current batch rather than an out-of-sample guarantee. In addition, the exchangeability of policy rollouts and MPPI samples is asserted rather than shown, and the online update of µj, σj from Aconformal breaks exchangeability across planning iterations. Eq. 8's weak-validity statement depends on a teaching schedule L that is never constructed, and the asymptotic guarantee is imported without verifying its conditions. These are substantial correctness risks, but they do not make the empirical method itself circular. Overall, the circularity is concentrated in the theoretical conformal core, while the empirical contribution retains independent content.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The central method depends on hand-chosen hyperparameters (alpha, G, beta, tau, N_pi, H) and on several unproved assumptions about exchangeability, a weak-teacher schedule, and bounded gradients. The invented uncertainty decomposition is conceptual and not independently evidenced.

free parameters (6)
  • alpha (conformal risk level) = 0.05
    Set by hand in Section IV; controls the quantile threshold in Eq. 6 and therefore the fraction of trajectories kept in A_conformal.
  • G (group size for GRPC) = 3
    Chosen for the group-relative policy objective in Eq. 12; no sensitivity analysis is reported.
  • beta (KL trust-region weight) = not reported
    Weight on the KL penalty in Eq. 13; no value is listed in the experimental setup.
  • tau (temperature) = not reported
    Used in MPPI reweighting Eq. 3 and softmax advantage Eq. 9; no value or schedule is specified.
  • N_pi (number of policy-prior trajectories) = 24
    Number of policy rollouts used as conformal calibration and prior trajectories; listed in Section IV hyperparameters.
  • planning horizon H = 3
    MPPI planning horizon used in experiments; directly limits the length of the conformal trajectory filtering.
assumptions (4)
  • ad hoc to paper The union of policy-rollout and MPPI trajectory scores is exchangeable at each planning step.
    Stated in Section III-B before Eq. 5 to invoke conformal prediction, but no argument is given for why trajectories sampled from two different distributions, the policy and the MPPI sampler, are exchangeable.
  • ad hoc to paper The TD value estimator V_theta is a weak teacher with a teaching schedule L satisfying lim_{k->infty} (n_k/n_{k-1}) = 1.
    Introduced in Section III-B solely to justify asymptotic weak validity (Eq. 8); no construction or empirical verification of such a schedule is provided.
  • domain assumption For the variance bound, ||grad_theta log pi_theta(a|s)|| = C and Q_theta, r_theta are bounded.
    Assumed in Section III-C to derive Var[grad L_softmax] <= Var[grad L_std-norm]; no proof or discussion of whether these hold for the learned networks.
  • standard math The TD-MPC framework with latent dynamics, MPPI, and value and reward estimators from Hansen et al. works as described.
    Background framework accepted from prior literature; the paper builds on it without re-deriving it.
invented entities (1)
  • Disjoint planning and policy uncertainty decomposition
    purpose: Provides the conceptual motivation for using conformal prediction for planning uncertainty and GRPC for policy uncertainty. The two components are not directly measured or validated as disjoint sources.
    The Introduction and Fig. 1 assert the decomposition, but the paper offers no experiment that isolates the two components or shows that they are independent. The ablation only removes whole modules and does not quantify either uncertainty type.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DoublyAware: Dual Planning and Policy Awareness for Temporal Difference Learning in Humanoid Locomotion." pith.science (2026). https://pith.science/paper/VMFF3LH6

@misc{pith2026250612095,
  author       = {Pith},
  title        = {Pith review of: DoublyAware: Dual Planning and Policy Awareness for Temporal Difference Learning in Humanoid Locomotion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VMFF3LH6}},
  note         = {Machine review of arXiv:2506.12095}
}
read the original abstract

Achieving robust robot learning for humanoid locomotion is a fundamental challenge in model-based reinforcement learning (MBRL), where environmental stochasticity and randomness can hinder efficient exploration and learning stability. The environmental, so-called aleatoric, uncertainty can be amplified in high-dimensional action spaces with complex contact dynamics, and further entangled with epistemic uncertainty in the models during learning phases. In this work, we propose DoublyAware, an uncertainty-aware extension of Temporal Difference Model Predictive Control (TD-MPC) that explicitly decomposes uncertainty into two disjoint interpretable components, i.e., planning and policy uncertainties. To handle the planning uncertainty, DoublyAware employs conformal prediction to filter candidate trajectories using quantile-calibrated risk bounds, ensuring statistical consistency and robustness against stochastic dynamics. Meanwhile, policy rollouts are leveraged as structured informative priors to support the learning phase with Group-Relative Policy Constraint (GRPC) optimizers that impose a group-based adaptive trust-region in the latent action space. This principled combination enables the robot agent to prioritize high-confidence, high-reward behavior while maintaining effective, targeted exploration under uncertainty. Evaluated on the HumanoidBench locomotion suite with the Unitree 26-DoF H1-2 humanoid, DoublyAware demonstrates improved sample efficiency, accelerated convergence, and enhanced motion feasibility compared to RL baselines. Our simulation results emphasize the significance of structured uncertainty modeling for data-efficient and reliable decision-making in TD-MPC-based humanoid locomotion learning.

Figures

Figures reproduced from arXiv: 2506.12095 by the authors.

Figure 1
Figure 1. Overview of DoublyAware: Disjoint uncertainty decomposi￾tion in TD-MPC frameworks and refinements for each component in planning and learning phases for robust humanoid locomotion. Planning uncertainty maps to aleatoric uncertainty, which is induced by the environment randomness together with the system dynamics (e.g., ground contact, observation noises, and multi-modal nature of feasible movement trajectories). Suc… view at source ↗
Figure 2
Figure 2. Uncertainty-Aware Planning for Humanoid Locomotion: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Policy-Aware Learning for Humanoid Locomotion: [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Episode Returns of DoublyAware and Baselines on H1–2 in Locomotion Tasks: DoublyAware achieves rapid convergence over others in standing, walking, sitting, navigating through poles, hurdling, and sliding tasks, while it performs worse in more complex tasks such as craw…
Figure 5
Figure 5. Figure 5: Qualitative Results of H1-2 in Locomotion Tasks: [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 24 canonical work pages

  1. [1]

    Learning humanoid locomotion with transformers,

    I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath, “Learning humanoid locomotion with transformers,” CoRR, 2023

    work page 2023

  2. [2]

    Real-world humanoid locomotion with reinforcement learning,

    ——, “Real-world humanoid locomotion with reinforcement learning,” Science Robotics, vol. 9, no. 89, p. eadi9579, 2024

    work page 2024

  3. [3]

    Model predictive control,

    B. Kouvaritakis and M. Cannon, “Model predictive control,” Switzer- land: Springer International Publishing , vol. 38, no. 13-56, p. 7, 2016

    work page 2016

  4. [4]

    Aleatoric and epistemic uncertainty with random forests,

    M. H. Shaker and E. H ¨ullermeier, “Aleatoric and epistemic uncertainty with random forests,” in International Symposium on Intelligent Data Analysis. Springer, 2020, pp. 444–456

    work page 2020

  5. [5]

    Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods,

    E. H ¨ullermeier and W. Waegeman, “Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods,”Machine learning, vol. 110, no. 3, pp. 457–506, 2021

    2021

  6. [6]

    Stochasticity in Motion: An Information-Theoretic Approach to Trajectory Prediction

    A. Distelzweig, A. Look, E. Kosman, F. Janjo ˇs, J. Wagner, and A. Val- ada, “Stochasticity in motion: An information-theoretic approach to trajectory prediction,” arXiv preprint arXiv:2410.01628 , 2024

    work page Pith review arXiv 2024

  7. [7]

    Learning Through Retrospection: Improving Trajectory Prediction for Automated Driving with Error Feedback

    S. Hagedorn, A. Distelzweig, M. Hallgarten, and A. P. Condurache, “Learning through retrospection: Improving trajectory prediction for au- tomated driving with error feedback,” arXiv preprint arXiv:2504.13785, 2025

    work page Pith review arXiv 2025

  8. [8]

    Temporal difference learning for model predictive control,

    N. A. Hansen, H. Su, and X. Wang, “Temporal difference learning for model predictive control,” in International Conference on Machine Learning. PMLR, 2022, pp. 8387–8406

    2022

Show all 40 references
  1. [9]

    Td-mpc2: Scalable, robust world models for continuous control,

    N. Hansen, H. Su, and X. Wang, “Td-mpc2: Scalable, robust world models for continuous control,” in The Twelfth International Conference on Learning Representations

  2. [10]

    Deep reinforce- ment learning in a handful of trials using probabilistic dynamics models,

    K. Chua, R. Calandra, R. McAllister, and S. Levine, “Deep reinforce- ment learning in a handful of trials using probabilistic dynamics models,” NeurIPS, 2018

    2018

  3. [11]

    Model-based offline planning,

    A. Argenson and G. Dulac-Arnold, “Model-based offline planning,” arXiv preprint arXiv:2008.05556 , 2020

    2008 arXiv

  4. [12]

    Improving td-mpc through policy constraint,

    H. Lin, P. Wang, J. Schneider, and G. Shi, “Improving td-mpc through policy constraint,” arXiv preprint arXiv:2502.03550 , 2025

    2025 arXiv

  5. [13]

    V ovk, A

    V . V ovk, A. Gammerman, and G. Shafer, Algorithmic learning in a random world. Springer, 2005, vol. 29

    2005

  6. [14]

    Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wuet al. , “Pushing the limits of mathematical reasoning in open language models,” arXiv preprint arXiv:2402.03300 , 2024

    2024 arXiv

  7. [15]

    Humanoid- bench: Simulated humanoid benchmark for whole-body locomotion and manipulation,

    C. Sferrazza, D.-M. Huang, X. Lin, Y . Lee, and P. Abbeel, “Humanoid- bench: Simulated humanoid benchmark for whole-body locomotion and manipulation,” arXiv preprint arXiv:2403.10506 , 2024

    2024 arXiv

  8. [16]

    Reinforcement learning for humanoid robotics,

    J. Peters, S. Vijayakumar, and S. Schaal, “Reinforcement learning for humanoid robotics,” in Proceedings of the third IEEE-RAS international conference on humanoid robots , 2003, pp. 1–20

    2003

  9. [17]

    Learning off-policy with online planning,

    H. Sikchi, W. Zhou, and D. Held, “Learning off-policy with online planning,” in CoRL, 2022

    2022

  10. [18]

    Conformal prediction in manifold learning,

    A. Kuleshov, A. Bernstein, and E. Burnaev, “Conformal prediction in manifold learning,” in Conformal and Probabilistic Prediction and Applications. PMLR, 2018, pp. 234–253

    2018

  11. [19]

    Conformal prediction with learned features,

    S. Kiyani, G. Pappas, and H. Hassani, “Conformal prediction with learned features,” arXiv preprint arXiv:2404.17487 , 2024

    2024 arXiv

  12. [20]

    Confor- mal prediction for semantically-aware autonomous perception in urban environments,

    A. Doula, T. G ¨udelh¨ofer, M. M ¨uhlh¨auser, and A. S. Guinea, “Confor- mal prediction for semantically-aware autonomous perception in urban environments,” in 8th Annual Conference on Robot Learning

  13. [21]

    Conformal prediction for uncertainty-aware planning with diffusion dynamics model,

    J. Sun, Y . Jiang, J. Qiu, P. Nobel, M. J. Kochenderfer, and M. Schwager, “Conformal prediction for uncertainty-aware planning with diffusion dynamics model,” Advances in Neural Information Processing Systems , vol. 36, pp. 80 324–80 337, 2023

    2023

  14. [22]

    Adaptive conformal prediction for motion planning among dynamic agents,

    A. Dixit, L. Lindemann, S. X. Wei, M. Cleaveland, G. J. Pappas, and J. W. Burdick, “Adaptive conformal prediction for motion planning among dynamic agents,” in Learning for Dynamics and Control Con- ference. PMLR, 2023, pp. 300–314

    2023

  15. [23]

    Safe planning in dynamic environments using conformal prediction,

    L. Lindemann, M. Cleaveland, G. Shim, and G. J. Pappas, “Safe planning in dynamic environments using conformal prediction,” IEEE Robotics and Automation Letters , vol. 8, no. 8, pp. 5116–5123, 2023

    2023

  16. [24]

    Safe perception-based control under stochastic sensor uncertainty using con- formal prediction,

    S. Yang, G. J. Pappas, R. Mangharam, and L. Lindemann, “Safe perception-based control under stochastic sensor uncertainty using con- formal prediction,” in 2023 62nd IEEE Conference on Decision and Control (CDC). IEEE, 2023, pp. 6072–6078

    2023

  17. [25]

    Conformal decision theory: Safe autonomous decisions from imperfect predictions,

    J. Lekeufack, A. N. Angelopoulos, A. Bajcsy, M. I. Jordan, and J. Malik, “Conformal decision theory: Safe autonomous decisions from imperfect predictions,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 11 668–11 675

    2024

  18. [26]

    Safe pomdp online planning among dynamic agents via adaptive conformal prediction,

    S. Sheng, P. Yu, D. Parker, M. Kwiatkowska, and L. Feng, “Safe pomdp online planning among dynamic agents via adaptive conformal prediction,” IEEE Robotics and Automation Letters , 2024

    2024

  19. [27]

    Conformal policy learning for sensorimotor control under distribution shifts,

    H. Huang, S. Sharma, A. Loquercio, A. Angelopoulos, K. Goldberg, and J. Malik, “Conformal policy learning for sensorimotor control under distribution shifts,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 16 285–16 291

    2024

  20. [28]

    Conformalized teleoperation: Confidently mapping human inputs to high-dimensional robot actions,

    M. Zhao, R. Simmons, H. Admoni, and A. Bajcsy, “Conformalized teleoperation: Confidently mapping human inputs to high-dimensional robot actions,” arXiv preprint arXiv:2406.07767 , 2024

    2024 arXiv

  21. [29]

    Stabilizing off- policy q-learning via bootstrapping error reduction,

    A. Kumar, J. Fu, M. Soh, G. Tucker, and S. Levine, “Stabilizing off- policy q-learning via bootstrapping error reduction,” NeurIPS, 2019

    2019

  22. [30]

    Conservative q-learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q-learning for offline reinforcement learning,” NeurIPS, 2020

    2020

  23. [31]

    A minimalist approach to offline reinforce- ment learning,

    S. Fujimoto and S. S. Gu, “A minimalist approach to offline reinforce- ment learning,” NeurIPS, vol. 34, pp. 20 132–20 145, 2021

    2021

  24. [32]

    Off-policy deep reinforcement learning without exploration,

    S. Fujimoto, D. Meger, and D. Precup, “Off-policy deep reinforcement learning without exploration,” in ICML, 2019

    2019

  25. [33]

    Advantage-weighted regression: Simple and scalable off-policy reinforcement learning,

    X. B. Peng, A. Kumar, G. Zhang, and S. Levine, “Advantage-weighted regression: Simple and scalable off-policy reinforcement learning,”arXiv preprint arXiv:1910.00177, 2019

    1910 arXiv

  26. [34]

    Extreme q-learning: Maxent rl without entropy,

    D. Garg, J. Hejna, M. Geist, and S. Ermon, “Extreme q-learning: Maxent rl without entropy,” arXiv preprint arXiv:2301.02328 , 2023

    2023 arXiv

  27. [35]

    Offline reinforcement learning with implicit q-learning,

    I. Kostrikov, A. Nair, and S. Levine, “Offline reinforcement learning with implicit q-learning,” arXiv preprint arXiv:2110.06169 , 2021

    2021 arXiv

  28. [36]

    Aggressive driving with model predictive path integral control,

    G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, “Aggressive driving with model predictive path integral control,” in 2016 IEEE international conference on robotics and automation (ICRA). IEEE, 2016, pp. 1433–1440

    2016

  29. [37]

    Trust region policy optimization,

    J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in International conference on machine learning. PMLR, 2015, pp. 1889–1897

    2015

  30. [38]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” in ICML, 2018

    2018

  31. [39]

    Imitation is not enough: Ro- bustifying imitation with reinforcement learning for challenging driving scenarios,

    Y . Lu, J. Fu, G. Tucker, X. Pan, E. Bronstein, R. Roelofs, B. Sapp, B. White, A. Faust, S. Whiteson et al. , “Imitation is not enough: Ro- bustifying imitation with reinforcement learning for challenging driving scenarios,” in 2023 IEEE/RSJ International Conference on Intelli...

    2023

  32. [40]

    Awac: Accelerating online reinforcement learning with offline datasets,

    A. Nair, A. Gupta, M. Dalal, and S. Levine, “Awac: Accelerating online reinforcement learning with offline datasets,” arXiv preprint arXiv:2006.09359, 2020

    2006 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.