Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Ego-centric Learning of Communicative World Models for Autonomous Driving

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In CALL, driving agents share latent world-model states and waypoints, roughly doubling multi-agent driving performance while cutting communication to about one-fiftieth of raw data.

desk verdict A plausible and novel combination of latent sharing and prediction-error-triggered communication, but with an invalid theoretical proof and an experimental claim confounded by an unshared waypoint input. read the letter →

arxiv 2506.08149 v1 pith:ZQLBN5SM submitted 2025-06-09 cs.RO cs.AI

Hang Wang , Dechen Gao , Junshan Zhang This is my paper
classification cs.ROcs.AI
keywords multi-agentreinforcementlearningworldmodelsautonomousdrivinglatentcommunicationintentionsharingpartialobservabilitynon-stationaritytrajectoryplanning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes CALL, a multi-agent reinforcement learning (MARL) method in which each vehicle learns a generative world model and compresses its bird's-eye-view (BEV) observation into a low-dimensional latent state and a latent intention — the vehicle's planned waypoints. Agents exchange these compressed representations with a modest set of nearby vehicles, and each agent widens its communication range only when its own multi-step prediction error exceeds a threshold, so the protocol adapts to how uncertain the world model actually is. The paper's central empirical claim is that this lightweight sharing roughly doubles the ego agent's return in CARLA local trajectory planning relative to a world-model baseline that sees only local observations, and that it learns faster than a baseline given full observations. The paper supports the claim with a bound on the multi-step prediction error that accumulates from recurrent-network generalization error, missing model-state information, and non-stationarity, and with a proposition showing that a small enough prediction error bounds the sub-optimality gap in the value function. Measured communication averages about 0.106 MB per 0.1 seconds in a 230-vehicle system, roughly one-fiftieth of sharing raw sensor input.

What carries the argument

The load-bearing object is the communicative world model itself: a recurrent state-space model (RSSM) in which an encoder maps each agent's bird's-eye-view observation to a latent state $z_{i,t}$, a recurrent sequence model maintains a hidden state $h_{i,t}$, and the planned waypoints are encoded as a latent intention $w_{i,t}$. The triple $[z_{i,t}, h_{i,t}, w_{i,t}]$ is the unit of communication, small enough that exchanging it with nearby agents costs roughly one-fiftieth of the bandwidth of raw observations. The decision mechanism is prediction-accuracy-driven sharing: the agent compares predicted latent states and intentions from the last $K$ steps against the actual encoded values, and when the error exceeds a threshold $c$, it increases its communication range by 5 meters. The analytical machinery is the error decomposition $\epsilon_{i,t+k} = (z_{i,t+k} - \bar{z}_{i,t+k}) + (\bar{z}_{i,t+k} - \hat{z}_{i,t+k})$, which splits the multi-step prediction error into RNN generalization error and epistemic error; the epistemic part further separates into a stationary gap from the missing model state $E_x$ and a non-stationary part from the total-variation model mismatch $E_P$. Theorem 1 bounds the accumulated error over the rollout horizon $k$, and Proposition 1 converts that bound into an upper bound on the sub-optimality gap of the value function.

What would settle it

Run the 150-vehicle CARLA experiment with the fleet split between two different world-model architectures (for example, different latent dimensions), which forces the Appendix F fallback of sharing semantic BEVs instead of latent triples. If the per-step bandwidth then rises back toward the 5 MB full-observation level while the reported 100% return gain over the local-observation baseline shrinks toward zero, the claim that lightweight latent sharing delivers the performance gain would be refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a world model's latent space can serve directly as a communication channel. Because the recurrent state-space model compresses each agent's bird's-eye-view observation into a small model state $x_{i,t} := [z_{i,t}, h_{i,t}]$ (latent state plus recurrent hidden state) and an intention $w_{i,t}$ (the planned waypoints), agents that share the same encoder-decoder architecture can exchange the triple $[z_{i,t}, h_{i,t}, w_{i,t}]$ and decode one another's states with their local decoders. CALL makes the sharing selective: each agent tracks its $K$-step prediction error $\epsilon_{i,t+k} := \|\hat{X}_{i,t} - X_{i,t}\|$ between predicted and actual latent states and intentions, and expands its communication range by 5 meters only when the error exceeds a threshold $c$. The paper reports that this yields roughly a 100% return improvement over the local-observation world-model baseline, faster learning than the full-observation baseline, and that intention sharing carries most of the benefit — ablations show waypoint sharing roughly doubles performance over latent-state sharing alone. The theoretical core is a decomposition of the multi-step lookahead prediction error into an RNN generalization term, an epistemic term from the missing model state (partial observability), and a total-variation term (non-stationarity); Theorem 1 bounds the accumulated error over the horizon, and Proposition 1 shows that a sufficiently small prediction error makes the shared information locally sufficient, bounding the sub-optimality gap in the ego agent's value function.

Load-bearing premise

The experiments assume all agents share the same encoder-decoder world model, so one vehicle can directly decode another's latent state (stated in Section 4); if world models differ, the paper's own Appendix F fallback is sharing semantic BEVs, which are far larger and erode the roughly one-fiftieth communication advantage that the main results depend on.

Editorial extensions

If this is right

  • If the roughly 100% return gain replicates, partial observability — not reward design or control difficulty — is the dominant bottleneck in the tested CARLA trajectory-planning regime, since decoding neighbors' latent states restores most of the missing information.
  • The one-fiftieth bandwidth figure makes vehicle-to-vehicle latent exchange viable under realistic wireless limits; the 230-agent measurement at about 0.106 MB per 0.1 seconds gives a concrete operating point for scaling.
  • Because sharing intentions (waypoints) alone roughly doubles performance over sharing latent states alone, future cooperative-driving stacks should prioritize exchanging planned trajectories, not only observations.
  • The adaptive range rule has a wide safe operating window (the paper reports good performance for thresholds $c \in [10, 80]$), so the method does not require careful tuning of the communication trigger.
  • The theory predicts prediction error accumulates with rollout horizon $k$, so CALL's benefit should grow on tasks that require longer lookahead; the paper's 30-step prediction comparisons are consistent with that prediction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the prediction-error rule does not actually require a generative decoder — any agent with a one-step differentiable latent predictor could run the same expand-when-uncertain protocol, so the sharing mechanism could transfer to model-free communication schemes the paper does not consider.
  • Beyond the paper: the homogeneity assumption is the fragile point, and a fix the paper leaves implicit is learning a small latent-to-latent translation map between heterogeneous world models, which would preserve the bandwidth advantage that the semantic-BEV fallback of Appendix F loses.
  • Beyond the paper: because full observation learns more slowly than CALL, the protocol implicitly acts as an information filter; a testable corollary is that in denser traffic the optimal communication range should stay small while the value of sharing intentions grows, which could be checked by sweeping vehicle density in CARLA.
  • Beyond the paper: the waypoint-ablation results suggest a cheaper variant — sharing only intentions and not latent states — might capture most of the gain at even lower bandwidth; the paper reports the components but never evaluates that combination.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CALL, a communicative world model for ego-centric multi-agent reinforcement learning in autonomous driving. Each agent learns a latent world model whose latent state and planned intentions (waypoints) are shared with neighboring agents, with the communication range adapted based on prediction error. The authors derive an upper bound on multi-step prediction error (Theorem 1) and a bound on the value-function sub-optimality gap (Proposition 1), and report CARLA experiments with 150 and 230/250 agents, claiming around 100% performance improvement over a local-observation DreamerV3 baseline and bandwidth of 0.106 MB versus 5.417 MB for full observation.

Significance. If the claims were established, CALL would offer a practically useful combination of world-model generalization and low-bandwidth communication for cooperative driving, with clear application value. The paper has concrete strengths: extensive CARLA experiments, multiple ablation studies, prediction visualizations, and an intent to release code. However, the theoretical derivation of Theorem 1 is invalid as written, the information-sharing gain is derived in a definitional rather than substantive way, and the headline experimental comparison omits a key control (waypoint input). These issues are load-bearing: the claimed contribution is not currently supported.

major comments (4)
  1. [Appendix B, Eqs. (7)–(10)] The proof of Theorem 1 treats the weight matrices W, U, V, A as scalar coefficients. For example, Eq. (8) writes (LhLzV W + LzV A)ϵ + LhLzV U Laϵ, but V W and V A are matrices, so the displayed inequality does not follow from Lipschitz continuity unless norms are taken and the recursion is formulated in a normed space. The subsequent recursion with N := LhLzV U La then uses powers N^h of a matrix and mixes scalars and matrices in the final bound; the argument is therefore not a valid scalar error recursion. This invalidates Theorem 1 as stated. The garbled definition of M in Section 3.1 ('M = BV BU (BW )k−1 BW −1') and the undefined quantity h in the term 2hBxEP reinforce that the expression was not carefully checked.
  2. [Section 3.1] The 'Prediction performance gain from information sharing' paragraph is definitional rather than derivational. It states that after acquiring shared information, the error terms Ex and EP are reduced by εx and εp, and concludes that the prediction error bound decreases by terms proportional to those reductions. Since Theorem 1 already has Ex and EP as arguments, this merely restates the theorem under an assumption of the conclusion. Proposition 1 is likewise a generic Lipschitz value-difference bound: it says that if the prediction error under a given information set is small, then the value gap is small. This is a standard template and does not establish that CALL's communication mechanism produces the required error reduction.
  3. [Section 4, Figs. 2(a) and 4(b)] The central empirical claim confounds two changes at once. CALL receives planned waypoints w_i,t as part of its input/action interface, while the 'Local obs.' baseline (DreamerV3) does not. Figure 4(b) shows that adding waypoints to a purely local observation baseline substantially improves learning even without any communication. No curve compares CALL against 'Local obs. + WP', so the reported 'around 100% performance improvement' in Figure 2(a) cannot be attributed to information sharing, prediction-error-driven communication, or the world model; it may be entirely a waypoint-input effect.
  4. [Section 4 and Appendix F] The lightweight-communication claim is not established for the general setting that the paper advertises. The 0.106 MB versus 5.417 MB bandwidth comparison is reported only for the homogeneous case where agents share the same encoder-decoder and can directly decode each other's latent states. Appendix F states that in the heterogeneous case agents instead share semantic BEVs, which are substantially larger, and no bandwidth or performance figures are given for that case. Moreover, no performance-matched communication baseline is provided: the bandwidth comparison does not show that the two settings operate at equal task performance, so the '1/50th bandwidth' conclusion is not supported by the experiments as reported.
minor comments (6)
  1. [Section 3.1] The definition of M is incomplete and should be corrected or removed: it reads 'M = BV BU (BW )k−1 BW −1' with no closing denominator.
  2. [Theorem 1] The quantity h in the term 2hBxEP is not defined, and the notation Ψh(δ, n) in Theorem 1 is inconsistent with Ψk(δ, n) used in the proof.
  3. [Appendix E.4] The manuscript states 250 vehicles in Appendix E.4 but 230 vehicles in Section 4; the counts should be reconciled.
  4. [Table 6 and Appendix E.4] The term 'LSI' is used for the CALL method in several places, whereas the main text uses 'CALL'; this inconsistency makes the ablation terminology hard to follow.
  5. [Figure 5] The caption refers to 'WM-MBRL' although the proposed method is CALL; the name should be updated or removed.
  6. [Reproducibility statement] The statement promises that source code and data 'will be made publicly available upon publication,' but no repository link or identifier is provided in the manuscript.

Circularity Check

1 steps flagged · score 6.0 of 10

CALL's theorized 'prediction gain from information sharing' is the assumed reduction of the error terms Ex and EP scaled by constants, so the headline theoretical benefit is definitional; the empirical 100%-gain claim is additionally confounded by a waypoint-input difference rather than isolating communication.

  1. self definitional [Section 3.1, 'Prediction performance gain from information sharing' (immediately after Theorem 1)]
    "More concretely, assume that after acquiring the information {zj,t, hj,t, wj,t}, j ∈ Gt ⊆ N, the error terms Ex and EP are reduced by εx, εp, respectively, then we can obtain that the prediction error can be improved by at least Pk j=1 N j 1 1/δ(N2εx + 2hBxεP) (ref. Theorem 1)."

    Here Ex and EP are, by definition in Theorem 1, the model-state gap and transition-model error that information sharing is supposed to reduce. The paper's claimed 'gain' is therefore not derived from any CALL-specific mechanism; it is the assumed reductions εx and εp multiplied by the theorem's Lipschitz constants and 1/δ factors. The sentence 'assume ... reduced by εx, εp' already contains the claimed benefit, and the displayed formula only restates that assumption in the theorem's notation. Theorem 1 is a generic multi-step Lipschitz error-accumulation bound and supplies no estimate, bound, or mechanism showing CALL actually achieves reductions εx and εp.

full rationale

Section 3.1 contains the clearest reductive step: the 'prediction performance gain from information sharing' is the assumed reduction in Ex and EP scaled by constants from Theorem 1; it is true by construction, not a derivation. Proposition 1 is also generic (any predictor with error Emax gives the same sub-optimality bound through the Lipschitz reward and LQ constants), so it adds no CALL-specific content, although it is a valid implication rather than a circular equation. No load-bearing self-citation or uniqueness-import pattern appears: the DreamerV3 and Think2Drive baselines are external, and no cited prior work by the same authors is invoked to force the design. There is, however, a separate experimental confound that limits the central empirical claim: Section 4 compares CALL, which includes planned waypoints wi,t, against a 'Local observation only' DreamerV3 baseline that lacks waypoints, while Figure 4(b) shows 'Local obs. + WP' already improves learning without communication; no curve isolates the communication effect from the waypoint-input effect. This is a missing control and a correctness risk, not a derivation-level circularity. The homogeneous-encoder assumption in Section 4 and Appendix F also narrows the bandwidth claim. Because one headline theoretical prediction reduces to its own assumption, the circularity score is 6 (partial), not higher, since the algorithm's adaptive communication rule and the experiments are not themselves derived from that formula.

Assumptions & free parameters 4 free parameters · 7 assumptions · 0 invented entities

The paper introduces no new physical entities such as particles or forces. The latent state and latent intention representations are algorithmic constructs built on existing world model components. The theoretical analysis relies on several unreported or questionable assumptions, and the algorithm itself has multiple free parameters whose values are not stated for the main experiments.

free parameters (4)
  • prediction accuracy threshold c = not reported in main experiments
    Algorithm 1 uses c to decide when to expand the communication range; Appendix G shows a performance/bandwidth tradeoff as c varies, but the value used in the main CARLA experiments is not stated.
  • communication range increment = 5 meters
    Hand-chosen step size for expanding the communication range when prediction error exceeds c (Algorithm 1).
  • reward weights w1...w6 = not specified
    The reward function in Section 4 and Appendix E.1 is a weighted sum of six components, but the numerical weights are not reported.
  • prediction evaluation window (past K steps) = not specified
    The algorithm compares predicted and actual latent states over the past K time steps, but K is not given for the experiments.
assumptions (7)
  • domain assumption Action is bounded and policy is La-Lipschitz (Assumption 1).
    Used in the proof of Theorem 1 to bound the effect of action perturbations on latent state predictions.
  • domain assumption RNN weight matrices W, U, V have bounded Frobenius norms (Assumption 2).
    Used to control the growth of prediction error in the recurrent network.
  • domain assumption Reward is Lr-Lipschitz (Assumption 3).
    Needed for the sub-optimality gap bound in Proposition 1.
  • domain assumption State space is compact and action space is finite (Assumption 4).
    MDP regularity used in the proof of Proposition 1 in Appendix C.
  • ad hoc to paper The world model RNN is trained on n i.i.d. samples from the same distribution as testing.
    Invoked for the generalization error lemma in Appendix B. In the autonomous driving setting, data are collected sequentially and are highly non-stationary, so the i.i.d. assumption is questionable.
  • ad hoc to paper Sharing latent information reduces Ex and EP by amounts εx and εp.
    The entire claim that information sharing improves prediction rests on this assumption. No mechanism or magnitude is derived; it is simply assumed to hold.
  • domain assumption All agents share the same encoder-decoder architecture in the experiments (homogeneous case).
    Required for agents to decode each other's shared latent states directly. Stated in Section 4 and Appendix F. The main experiments only evaluate this case, so the bandwidth advantage is not demonstrated for heterogeneous agents.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ego-centric Learning of Communicative World Models for Autonomous Driving." pith.science (2026). https://pith.science/paper/ZQLBN5SM

@misc{pith2026250608149,
  author       = {Pith},
  title        = {Pith review of: Ego-centric Learning of Communicative World Models for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZQLBN5SM}},
  note         = {Machine review of arXiv:2506.08149}
}
read the original abstract

We study multi-agent reinforcement learning (MARL) for tasks in complex high-dimensional environments, such as autonomous driving. MARL is known to suffer from the \textit{partial observability} and \textit{non-stationarity} issues. To tackle these challenges, information sharing is often employed, which however faces major hurdles in practice, including overwhelming communication overhead and scalability concerns. By making use of generative AI embodied in world model together with its latent representation, we develop {\it CALL}, \underline{C}ommunic\underline{a}tive Wor\underline{l}d Mode\underline{l}, for MARL, where 1) each agent first learns its world model that encodes its state and intention into low-dimensional latent representation with smaller memory footprint, which can be shared with other agents of interest via lightweight communication; and 2) each agent carries out ego-centric learning while exploiting lightweight information sharing to enrich her world model, and then exploits its generalization capacity to improve prediction for better planning. We characterize the gain on the prediction accuracy from the information sharing and its impact on performance gap. Extensive experiments are carried out on the challenging local trajectory planning tasks in the CARLA platform to demonstrate the performance gains of using \textit{CALL}.

Figures

Figures reproduced from arXiv: 2506.08149 by the authors.

Figure 1
Figure 1. Illustration of CALL: Ego-centric learning in the two-agent case. Moreover, a RSSM Hafner et al. (2023; 2020) model is used to capture the context information of the cur￾rent observation in the latent space by incorporating the hidden state in the encoder, i.e., Encoder: zi,t ∼ qϕ(zi,t|hi,t, oi,t, Ti,t), For brevity, we denote the concatenation of hi,t and zi,t as the model state xi,t := [hi,t, zi,t] ∈ X . Then a re… view at source ↗
Figure 2
Figure 2. (a) RL performance comparison between two settings: Local observation only (no commu [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Multi-step predictions results. The first five frames are used as context input; and the [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: (a-b) Ablation studies on the impact of waypoints: ‘Local obs. + WP’ represents the case [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: The bandwidth requirements (MB) of WM-MBRL in one testing. The red line is the [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]
Figure 6
Figure 6. Figure 6: BEV encoding. In this case, each vehicle will generate the encoded messages based on its [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: BEV decoding. In the setting where agents share the same encoder-decoder in the WM, the [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]
Figure 8
Figure 8. Figure 8: Examples of BEV representations (top left square). [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: The learning performance comparison and the ablation study on the model state. [PITH_FULL_IMAGE:figures/full_fig_p029_9.png]
Figure 10
Figure 10. Figure 10: The ablation studies on the waypoints sharing. [PITH_FULL_IMAGE:figures/full_fig_p029_10.png]
Figure 11
Figure 11. Figure 11: Learning Speed Comparison. It can be seen that at around 230k, the full observation with [PITH_FULL_IMAGE:figures/full_fig_p029_11.png]
Figure 12
Figure 12. Figure 12: Evaluation curves in three settings: Local observation, Full observation and [PITH_FULL_IMAGE:figures/full_fig_p030_12.png]
Figure 13
Figure 13. Figure 13: The comparison of the BEV multi-step prediction results with different information [PITH_FULL_IMAGE:figures/full_fig_p031_13.png]
Figure 14
Figure 14. Figure 14: Comparison of underlying true BEV and LSI BEV. [PITH_FULL_IMAGE:figures/full_fig_p031_14.png]
Figure 15
Figure 15. Figure 15: Evaluation of World Model’s Generalization Capability in the four-lane road section with [PITH_FULL_IMAGE:figures/full_fig_p032_15.png]
Figure 16
Figure 16. Figure 16: Evaluation of World Model’s Generalization Capability in the unseen environment. [PITH_FULL_IMAGE:figures/full_fig_p032_16.png]
Figure 17
Figure 17. Figure 17: Heterogeneous World Model Setting. In this setting, agents are equiped with different [PITH_FULL_IMAGE:figures/full_fig_p032_17.png]
Figure 18
Figure 18. Figure 18: An illustration of CALL in a two-agent case: Each agent encodes high-dimensional sensory inputs and planned waypoints into low-dimensional latent state and latent intent, which can be shared via lightweight communications (e.g., red dashed arrow) and used as inputs to…
Figure 19
Figure 19. Figure 19: The impact of parameter c on average return and the bandwidth requirements for commu￾nication. stable error patterns, with CALL typically maintaining intermediate error levels between the other two methods. In Figures 20(b) and 21(b), we consider the 250 agents case. …
Figure 20
Figure 20. Figure 20: The comparison of accumulation error in the 30 steps predictions. [PITH_FULL_IMAGE:figures/full_fig_p034_20.png]
Figure 21
Figure 21. Figure 21: The comparison of single-step prediction errors. [PITH_FULL_IMAGE:figures/full_fig_p035_21.png]
Figure 22
Figure 22. Figure 22: The comparison of prediction errors and communication ranges in 150 agents case. [PITH_FULL_IMAGE:figures/full_fig_p036_22.png]
Figure 23
Figure 23. Figure 23: The comparison of prediction errors and communication ranges in 250 agents case. [PITH_FULL_IMAGE:figures/full_fig_p037_23.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Comprehensive Survey on World Models for Embodied AI

    cs.CV 2025-10 conditional novelty 6.0 of 10

    A unified three-axis taxonomy — functionality, temporal modeling, spatial representation — organizes the world-model literature for embodied AI.

  2. GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspective

    cs.AI 2025-07 unverdicted novelty 3.0 of 10

    A position paper claiming that generative-AI agents that model and predict multi-agent dynamics will replace today's reactive MARL approaches.

Reference graph

Works this paper leans on

35 extracted references · 18 canonical work pages · cited by 2 Pith papers

  1. [1]

    Deep learning using rectified linear units (relu).arXiv preprint arXiv:1803.08375,

    Abien Fred Agarap. Deep learning using rectified linear units (relu).arXiv preprint arXiv:1803.08375,

    arXiv

  2. [3]

    With Assumption Assumption 3 holds, we obtain the following bound, rl − ˆrl ≤Lr(1 + Lπ)ϵt+l Qt+L − ˆQt+L ≤LQ(1 + Lπ)ϵt+L, where LQ := Lr 1−γ

    The policy π is Lπ-Lipschitz, i.e., dA(π(·|s) − π(·|s′)) ≤ LπdS(s, s′). With Assumption Assumption 3 holds, we obtain the following bound, rl − ˆrl ≤Lr(1 + Lπ)ϵt+l Qt+L − ˆQt+L ≤LQ(1 + Lπ)ϵt+L, where LQ := Lr 1−γ . Then we have the upper bound for the sub-optimality gap as, Ea h Q(st, at) − ˆQ(ot, at) i ≤Ea∼π "L−1X l=0 γlLr(1 + Lπ)ϵt+l # + γLEa∼π [LQ(1 + ...

    work page 2019

  3. [4]

    Model-free deep reinforcement learning for urban autonomous driving

    12 Jianyu Chen, Bodi Yuan, and Masayoshi Tomizuka. Model-free deep reinforcement learning for urban autonomous driving. In 2019 IEEE intelligent transportation systems conference (ITSC), pp. 2765–2771. IEEE, 2019a. Jianyu Chen, Shengbo Eben Li, and Masayoshi Tomizuka. Interpretable end-to-end urban autonomous driving with latent deep reinforcement learnin...

    work page 2019

  4. [5]

    The BEV representation can be learnt by using algorithms such as BevFusion Liu et al

    Learning BEV Representation. The BEV representation can be learnt by using algorithms such as BevFusion Liu et al. (2023), which is capable of unifying the cameras, LiDAR, Radar data into a BEV representation space. In our experiment, we leverage the privileged information provided by CARLA Dosovitskiy et al. (2017), such as location information and map t...

    work page 2023

  5. [7]

    We provide the detailed demo videos and images in the supplementary materials

    Demo videos and images. We provide the detailed demo videos and images in the supplementary materials. Report of Standard Deviation in Figures. Note that all the learning curves presented in this work are smoothed by using exponential moving average with smoothing factor 0.72, with the shaded area to be the standard deviation. We use the smoothing algorit...

    work page 2018

  6. [8]

    E.2 W ORLD MODEL TRAINING We use Dreamer v3 Hafner et al

    In this section, we summarize the experiment results in the 250 vehicles systems. E.2 W ORLD MODEL TRAINING We use Dreamer v3 Hafner et al. (2023) structure, i.e., encoder-decoder, RSSM Hafner et al. (2019), to train the world model and adopt the large model for all experiments with dimension summarized in Table

    work page 2023

  7. [11]

    Mastering diverse domains through world models

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104,

    arXiv

  8. [12]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114,

    arXiv

Show all 35 references
  1. [13]

    Think2drive: Efficient reinforcement learning by thinking in latent world model for quasi-realistic autonomous driving (in carla-v2)

    Qifeng Li, Xiaosong Jia, Shaobo Wang, and Junchi Yan. Think2drive: Efficient reinforcement learning by thinking in latent world model for quasi-realistic autonomous driving (in carla-v2). arXiv preprint arXiv:2402.16720,

    arXiv

  2. [14]

    Efficient multi-agent reinforcement learning by planning

    Qihan Liu, Jianing Ye, Xiaoteng Ma, Jun Yang, Bin Liang, and Chongjie Zhang. Efficient multi-agent reinforcement learning by planning. arXiv preprint arXiv:2405.11778,

    arXiv

  3. [15]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation

    Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L Rus, and Song Han. Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation. In 2023 IEEE international conference on robotics and automation (ICRA), pp. 2774–2781. IEEE,

    2023

  4. [16]

    Trajectory planning for autonomous vehicles using hierarchical reinforcement learning

    Kaleb Ben Naveed, Zhiqian Qiao, and John M Dolan. Trajectory planning for autonomous vehicles using hierarchical reinforcement learning. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 601–606. IEEE,

    2021

  5. [18]

    Scalable multi-agent reinforcement learning for networked systems with average reward

    Guannan Qu, Yiheng Lin, Adam Wierman, and Na Li. Scalable multi-agent reinforcement learning for networked systems with average reward. Advances in Neural Information Processing Systems, 33:2074–2086,

  6. [19]

    Car2x-based perception in a high-level fusion architecture for cooperative perception systems

    Andreas Rauch, Felix Klanner, Ralph Rasshofer, and Klaus Dietmayer. Car2x-based perception in a high-level fusion architecture for cooperative perception systems. In 2012 IEEE Intelligent Vehicles Symposium, pp. 270–275. IEEE,

    2012

  7. [20]

    Benchmarking model-based reinforcement learning

    Tingwu Wang, Xuchan Bao, Ignasi Clavera, Jerrick Hoang, Yeming Wen, Eric Langlois, Shunshi Zhang, Guodong Zhang, Pieter Abbeel, and Jimmy Ba. Benchmarking model-based reinforcement learning. arXiv preprint arXiv:1907.02057,

    1907 arXiv

  8. [21]

    Leveraging world model disentanglement in value-based multi-agent reinforcement learning

    Zhizun Wang and David Meger. Leveraging world model disentanglement in value-based multi-agent reinforcement learning. arXiv preprint arXiv:2309.04615,

  9. [22]

    Statistical machine learning in model predictive control of nonlinear processes

    Zhe Wu, David Rincon, Quanquan Gu, and Panagiotis D Christofides. Statistical machine learning in model predictive control of nonlinear processes. Mathematics, 9(16):1912,

    1912

  10. [23]

    An overview of multi-agent reinforcement learning from game theoretical perspective

    Yaodong Yang and Jun Wang. An overview of multi-agent reinforcement learning from game theoretical perspective. arXiv preprint arXiv:2011.00583,

    2011 arXiv

  11. [24]

    Combo: Compositional world models for embodied multi-agent cooperation

    Hongxin Zhang, Zeyuan Wang, Qiushi Lyu, Zheyuan Zhang, Sunli Chen, Tianmin Shu, Yilun Du, and Chuang Gan. Combo: Compositional world models for embodied multi-agent cooperation. arXiv preprint arXiv:2404.10775,

    arXiv

  12. [25]

    Multi-agent reinforcement learning: A selective overview of theories and algorithms

    Kaiqing Zhang, Zhuoran Yang, and Tamer Ba¸ sar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of reinforcement learning and control, pp. 321– 384, 2021a. Weinan Zhang, Xihuai Wang, Jian Shen, and Ming Zhou. Model-based multi-agen...

    arXiv

  13. [27]

    Chen et al

    Discount factor ai,t Action chosen by agent i at time t at Joint action [a1,t, · · ·, aN,t] oi,t ∈ Ωi Local observation of agent i at time t ri,t Reward received by agent i at time t πi Policy of agent i zi,t ∈ Z Latent state representation hi,t Hidden state from RNN xi,t = [h...

    2019

  14. [28]

    • ¯zt+k: the prediction generated by RNN if the Locally Sufficient Information (LSI) is employed as input

    when using the agent’s local observation, i.e., xi,t. • ¯zt+k: the prediction generated by RNN if the Locally Sufficient Information (LSI) is employed as input. To further analyze the impact of LSI, we decompose the prediction error into two parts: (1) General- ization error i...

    2023

  15. [29]

    L−1X l=0 γlr(st+l, at+l, a′ t+l) + γLQt−1(st+L, at+L, a′ t+L) # − Ea∼π

    Assume the training and testing datasets are drawn from the same distribution. Then with probability at least 1 − σ, the generalization error in terms of the expected loss function has the upper bound as follows, E[f (zi,t+k − ¯zi,t+k] ≤ ln + 3 s log 2 δ 2n + O Lrdy dM Ba(1 + ...

    2021

  16. [34]

    Return normalization limit L 1 Return normalization decay — 0.99 Actor entropy scale η 3 · 10−4 Learning rate — 3 · 10−5 Adam epsilon ϵ 10−5 Gradient clipping — 100 Table 5: Dreamer v3 hyper parameters Hafner et al. (2023). World Model Training.The world model is implemented a...

    2023

  17. [35]

    to learn the environment dynamics, encoder, reward, continuity and encoder-decoder. We list the equations from the RSSM mode as follows: RSSM    Sequence model: ht = fϕ(ht−1, zt−1, at−1) Encoder: zt ∼ qϕ(zt|ht, xt) Dynamics predictor: ˆzt ∼ pϕ(ˆzt|ht) Reward predictor: ˆrt ...

    2023

  18. [1991]

    The dynamics of reinforcement learning in cooperative multiagent systems

    Caroline Claus and Craig Boutilier. The dynamics of reinforcement learning in cooperative multiagent systems. AAAI/IAAI, 1998(746-752):2,

    1998

  19. [2007]

    A R ELATED WORK Model-based MARL

    17 Appendix. A R ELATED WORK Model-based MARL. Model based RL in single-agent setting has shown promising results in both theoretical analysis and practical experiments, especially in terms of the sampling efficiency Moerland et al. (2023); Yarats et al. (2021); Kaiser et al. ...

    2023

  20. [2008]

    Independent learning in stochastic games

    Asuman Ozdaglar, Muhammed O Sayin, and Kaiqing Zhang. Independent learning in stochastic games. arXiv preprint arXiv:2111.11743,

    arXiv

  21. [2009]

    Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533,

    Christian Schroeder de Witt, Tarun Gupta, Denys Makoviichuk, Viktor Makoviychuk, Philip HS Torr, Mingfei Sun, and Shimon Whiteson. Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533,

    2011 arXiv

  22. [2016]

    World models

    David Ha and Jürgen Schmidhuber. World models. arXiv preprint arXiv:1803.10122,

    arXiv

  23. [2018]

    Deepmind lab

    Charles Beattie, Joel Z Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, et al. Deepmind lab. arXiv preprint arXiv:1612.03801,

    arXiv

  24. [2019]

    Mastering atari with discrete world models

    Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. arXiv preprint arXiv:2010.02193,

    2010 arXiv

  25. [2020]

    Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst

    Mayank Bansal, Alex Krizhevsky, and Abhijit Ogale. Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst. arXiv preprint arXiv:1812.03079,

    arXiv

  26. [2021]

    End-to- end autonomous driving: Challenges and frontiers

    Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End-to- end autonomous driving: Challenges and frontiers. arXiv preprint arXiv:2306.16927,

    arXiv

  27. [2023]

    Cooper: Cooperative perception for connected autonomous vehicles based on 3d point clouds

    Qi Chen, Sihai Tang, Qing Yang, and Song Fu. Cooper: Cooperative perception for connected autonomous vehicles based on 3d point clouds. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pp. 514–524. IEEE, 2019b. Valliappa Chockalingam, Tegg T...

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.