{"id":"b062b83e-3db6-4b57-be87-eb730d438671","arxiv_id":"2506.21976","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A single diffusion model predicts agent and traffic-light states, including which agents are present, to enable closed-loop trip-level traffic simulation over kilometer-scale maps.","lead":"This paper introduces SceneDiffuser++, a diffusion-based world model that generates long, city-scale traffic simulations with agents entering and leaving the scene and traffic lights changing state. It aims to let autonomous vehicle software be tested on many synthetic miles rather than limited logged drives.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Validity is trained as logged-AV visibility but deployed under arbitrary simulated routes, with no mechanism tying predicted validity to the simulated AV's sensor geometry; this undermines the occlusion/spawn-removal claims and the validity-based realism metrics.","rationale":"The reader's weakest_assumption and my independent reading converge: the validity channel is the linchpin of the paper's novelty (dynamic agent generation, occlusion reasoning) and of its evaluation (all validity-based metrics). The paper does provide genuine internal support for the sparse-tensor training mechanism—Table 3 shows soft clipping is clearly more effective than hard clipping or no clipping—and the traffic-light transition matrix is a useful non-validity-dependent check. But the validity-semantics problem is not an ablation-level issue; it is a mismatch between the training label definition (logged AV visibility) and the deployment setting (arbitrary simulated AV route). Soft clipping stabilizes training, but it cannot fix what the validity label means at inference time. The point A-to-B wording is also unsupported without goal conditioning, but I regard it as secondary: even with a route-conditioned planner, the validity channel would still need to be tied to the simulated AV's actual sensor geometry. The proposed geometric-consistency test would settle whether the model has learned occlusion reasoning or merely a plausible validity prior. I therefore agree with the reader's conditional verdict, and no additional verdict change beyond CONDITIONAL is needed.","tokens_in":17052,"tokens_out":7146,"duration_ms":85245,"concrete_test":"Using the trained checkpoint (or a retrained model if the checkpoint is withheld), run 60-second route-unconditioned rollouts with SceneDiffuser++ as both planner and world model. For every timestep and every predicted-valid agent, compute geometric visibility from the simulated AV: check that the agent is within the detector range used to construct WOMD labels and that the line of sight to the agent is not blocked by roadgraph/building geometry. Compare the resulting visibility-based validity pattern to SceneDiffuser++'s predicted validity channel and to the logged validity distribution. If the fraction of predicted-valid agents that are geometrically visible from the simulated AV differs substantially from the corresponding logged fraction, the validity channel is not sensor-consistent under route divergence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central technical novelty is the learned validity channel introduced in Sec. 3 ('Learning Sparse Tensors'), and the paper defines validity in Appendix A.4 as whether an agent appears in the logged AV's detection output. This makes ground-truth validity a sensor-relative quantity: it depends on the logged AV's trajectory, detection range, and occlusion. During training, positions are normalized by the AV's ego pose and the denoiser is trained to reproduce this logged visibility. During inference, however, the model is deployed in route-unconditioned rollouts (Appendix A.4), where the simulated AV follows a different, unplanned route. The soft-clipping sampler (Fig. 3, right) multiplies predicted feature values by predicted validity, but nothing in the loss, architecture, or sampler constrains validity to be geometrically consistent with the simulated AV's actual sensor frustum. The model sees the simulated AV's past trajectory, so it might learn an implicit proxy, but there is no raycast, sensor model, or geometric constraint guaranteeing that a predicted-valid agent is visible from the simulated AV. The generated validity channel can therefore encode 'agent exists on the map' rather than 'agent is visible to the simulated AV.' This conflation breaks the claimed occlusion-reasoning capability and invalidates the validity-centric evaluation metrics (# valid agents, entering/exiting agents, entering/exiting distances, and the composite gains over IDM and SceneDiffuser), since those metrics compare a generated label of uncertain physical meaning to logged sensor-derived visibility. The point A-to-B claim is also weakened: Appendix A.4 states SceneDiffuser++ does not use or ingest a goal location, so the route-unconditioned rollout is the only regime actually evaluated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SceneDiffuser++, a diffusion-based generative world model for city-scale traffic simulation. The model jointly predicts, from a single denoising objective, agent trajectories, a validity channel that encodes agent presence/visibility, and traffic light states, via an autoregressive rollout over a sparse scene tensor. The authors introduce soft clipping at inference to stabilize the generation of sparse tensors, a multi-tensor architecture to jointly handle agents and traffic lights with different feature dimensions, and a map-extended version of the Waymo Open Motion Dataset (WOMD-XLMap) for trip-level simulation. They evaluate realism using JS divergence between histograms of simulated and logged metrics (agent counts, entering/exiting distances, offroad and collision rates, speeds, traffic-light transitions), reporting quantitative gains over IDM and SceneDiffuser baselines on 60s rollouts.","tokens_in":17350,"tokens_out":4081,"duration_ms":45854,"significance":"If the central claims are upheld, the paper would make a notable contribution: a single end-to-end model that handles scene generation, dynamic agent insertion/removal, occlusion, and traffic-light simulation without task-specific heads or heuristics. The sparse-tensor learning method is an interesting technical idea, and the joint modeling of agents and traffic lights is a useful step beyond prior work. The paper also provides an enlarged dataset and a nontrivial evaluation protocol for long-horizon simulation, which is valuable to the community. However, the claim of 'point A-to-B simulation' is not actually exercised in the experiments, and the validity channel, which is central to the new capabilities, has a train/inference semantic mismatch. These issues currently prevent the paper's strongest claims from being supported.","major_comments":[{"comment":"The abstract and Section 1 claim that SceneDiffuser++ is capable of 'point A-to-B simulation on a city scale', but Appendix A.4 explicitly states: 'SceneDiffuser and SceneDiffuser++ do not use goal-oriented routing; in other words, they do not use or ingest a goal location in any way, shape, or form.' Figure 1 depicts a 'trip end' star but no conditioning mechanism is described or tested. The route-unconditioned rollouts used in Section 4.1 do not implement point-to-point trips, so this central claim is unsupported as stated. Either the model must be extended with goal/route conditioning, or the paper should be reframed around route-unconditioned world-model simulation and the point-to-point claim removed from the abstract and contributions.","section":"Abstract and Appendix A.4"},{"comment":"The validity channel is trained to reproduce whether an agent appears in the logged AV's detection output, and all positions are normalized by the logged AV's ego pose (Section 3). In route-unconditioned rollouts, the simulated AV follows an arbitrary path that diverges from the logged trajectory, and nothing in the loss, architecture, or sampling procedure ties predicted validity to the simulated AV's actual sensor frustum or detection range. The generated validity can therefore encode 'agent exists on the map' rather than 'agent is visible to this AV'. This undermines the paper's occlusion-reasoning claims and makes the validity-based metrics (# valid agents, entering/exiting agents and distances) an unvalidated proxy for sensor-relative realism. The authors should either constrain validity at inference with the simulated AV's sensor geometry (e.g., raycasting or frustum culling) or provide evidence that the learned validity remains consistent with the simulated AV's visibility.","section":"Appendix A.4 (Validity Definition) and Section 3 (Learning Sparse Tensors)"},{"comment":"The evaluation relies on JS divergence between histograms of simulated and logged metric values, but the paper does not specify histogram bin width or number of bins, report error bars or confidence intervals, state the number of independent rollout seeds, or validate that the divergence is sensitive to the sample sizes used. The numbers in Table 1, such as 0.3132 vs. 0.2206 for # Valid Agents, may not be statistically distinguishable. In addition, the Composite score averages metrics with different units and scales without justification. Without these details, the headline 'state-of-the-art trip-level simulation realism' is not established. Adding standard errors over seeds and explicit histogram construction would make the comparison interpretable.","section":"Section 4.1 (Metrics)"},{"comment":"Several comparison cells are missing for the IDM and SceneDiffuser baselines: Entering and Exiting Distances are not reported for these world models, and Traffic Light Violation and Traffic Light Transition are not reported for any baseline. The text states that SceneDiffuser++ 'achieves significantly better performance in all metrics that relate to agent insertion and removal', but two of the four insertion/removal metrics lack baseline values. Moreover, because IDM and SceneDiffuser never insert agents, their validity distributions are degenerate by construction, which can inflate or deflate JS divergence in ways not discussed. The comparison should either report the missing values (with a principled definition for non-inserting baselines) or restrict the superiority claim to the metrics actually measured against both baselines.","section":"Table 1 and Section 4.2"},{"comment":"The text in Section 3 states that imputing invalid values with zeros 'cannot work', yet Table 3 includes a 'No Clipping' row described as 'a model trained to directly predict invalid agents' features to be 0', which yields a lower JS divergence for # Valid Agents (0.2426) than Soft Clipping (0.3053) and a comparable # Entering Agents (0.2035 vs. 0.2120). The claim that this approach 'cannot work' is internally inconsistent with the reported data. The ablation should present the full trade-off (e.g., no clipping is better on some agent-count metrics, worse on collision/offroad/TL), and the choice of soft clipping should be justified by the metric priorities of the target use case rather than by a blanket statement.","section":"Section 4.3, Table 3"}],"minor_comments":[{"comment":"The legend in the bottom plot uses 'SceneDiffusion', which should be 'SceneDiffuser' for consistency with the rest of the paper.","section":"Figure 7"},{"comment":"The loss weight w is described only as 'a loss weighting term'; its exact form (including the loss mask described later in the same section) should be specified in the equation or immediately after, since it is central to the sparse-tensor training.","section":"Section 3, Eq. (2)"},{"comment":"The 'Ground-truth Log' panel covers only 91 steps but the predicted panels show 600 steps; the caption should state this difference explicitly to avoid implying direct temporal alignment beyond step 91.","section":"Figure 6"},{"comment":"The phrase 'Average Speed likelihood' appears to mix a distributional metric with a probabilistic notion; consider rewording to 'Average Speed distribution realism'.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's central technical ideas are interesting and the appendices are unusually candid about the route-unconditioned setup, which is to the authors' credit. The main risk is that the abstract and contributions overstate point-to-point capability, and the validity semantic mismatch is not merely cosmetic: it bears directly on the claimed occlusion reasoning. The evaluation would also need error bars and full baseline cells before I could assess whether the reported improvements are significant. I would not recommend rejection, as these issues are addressable with reframing and targeted additional analyses within the paper's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuine, incremental advance for learned traffic simulation, not a paradigm break. The sparse-tensor validity channel with soft clipping is a clever and well-ablated contribution, and jointly modeling traffic lights with a multi-tensor denoiser fills a real gap. The paper also deserves credit for explicitly tackling agent spawn/removal and occlusion, which most learned simulators ignore, and for being honest about its own weaknesses like offroad and collision rates.\n\nThe soft spots are real but not fatal. First, the 'point A-to-B' claim in the abstract is not supported by the experiments. Appendix A.4 states the model does not use or ingest a goal location, and the evaluation is route-unconditioned. That is a sensible choice for studying world-model realism, but the paper must not claim A-to-B capability unless the planner is goal-conditioned and the world model is actually tested under a goal. This is an overclaim that should be fixed.\n\nSecond, the validity channel. Validity is defined as whether an agent appears in the logged AV's detection output, so it is a sensor-relative quantity. During route-unconditioned rollouts the simulated AV follows a different path, and nothing in the loss or architecture ties predicted validity to the simulated AV's current sensor frustum. The model may be learning a proxy like 'agent exists near the AV' rather than 'agent is visible.' That would weaken the occlusion-reasoning interpretation and the validity-based metrics, which compare simulated validity distributions against logged visibility. The soft-clipping ablation is convincing and the qualitative results look good, but the semantics of validity under arbitrary routes are under-defined. I would like to see either a simple geometric sensor model or an analysis showing how predicted validity correlates with the simulated AV's pose and line-of-sight.\n\nThird, the evaluation. JS divergence is a reasonable metric for trip-level realism, but the paper gives no error bars and minimal histogram details, so it is hard to tell whether the reported gains are stable. The long-horizon rollouts and replan-frequency ablation are good practice, but the comparison to IDM and SceneDiffuser would be stronger with confidence intervals.\n\nOn balance, the core method is solid and the paper is clearly written. The overclaim in the abstract is easy to fix, and the validity concern is worth investigating but does not sink the approach. I would send this to a serious venue for review, and I would likely conditionally accept after the authors clarify the A-to-B framing and validity semantics, and add error bars.","headline":"A real step forward for trip-level traffic simulation with a clever sparse-tensor validity trick, but the 'point A-to-B' claim outruns the experiments and the validity channel's sensor-relative semantics deserve closer scrutiny.","tokens_in":17926,"tokens_out":3919,"would_cite":true,"duration_ms":41478,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a single diffusion-based generative world model, trained end-to-end on one denoising loss, can simulate a full trip from point A to point B at city scale, jointly generating agent trajectories, agent validity…","keywords":["traffic simulation","diffusion model","world model","autoregressive rollout","sparse tensors","agent validity","traffic lights","Waymo Open Motion Dataset"],"falsifier":"Replay 60-second SceneDiffuser++ rollouts with a fixed route-unconditioned planner, then ray-cast from the simulated ego pose against the map geometry to compute the physically visible agent set at each step; if the model's predicted validity (or its entering and exiting events) matches ray-cast visibility no better than a baseline that ignores map occlusion, the claim that the validity channel encodes occlusion and visibility would fail.","tokens_in":16893,"feed_emoji":"🚗","tokens_out":6255,"duration_ms":67080,"temperature":0.7,"pith_summary":"SceneDiffuser++ sets out to show that one end-to-end generative model can replace the usual stack of specialised traffic simulators: given a large map, a start point, and an autonomous vehicle stack, it should populate the scene, animate vehicles and pedestrians, decide when agents appear and disappear, reason about occlusion, and set traffic light states for a trip lasting minutes rather than seconds. The paper claims all of this can be done by a single diffusion model trained on a single denoising loss, then rolled out autoregressively. To support the claim, the authors build a sparse scene tensor in which every agent and traffic light carries a validity channel indicating its presence in the ego vehicle's detection output, so spawning, removal, and occlusion become one prediction problem rather than separate modules. On a map-extended version of the Waymo Open Motion Dataset with kilometer-scale regions, they measure realism by comparing simulated and logged distributions over agent counts, entry and exit distances, speeds, offroad and collision rates, and traffic-light transitions, and report lower divergence than IDM and SceneDiffuser across most trip-level metrics. A sympathetic reader would take this as evidence that trip-level simulation of a generative city is feasible as one learned world model.","feed_headline":"One diffusion model drives city-scale point-A-to-B traffic simulation","feed_subtitle":"It also predicts when agents appear, vanish, or are occluded, plus traffic-light states, over 60-second trips on a city map.","key_machinery":"The load-bearing object is the multi-tensor scene representation $X = \\{x_{\\mathrm{agent}}, x_{\\mathrm{light}}\\}$, where each $x_i \\in \\mathbb{R}^{E_i \\times T \\times D_i}$ stacks $E_i$ elements over $T$ timesteps and each element's last feature is a validity bit. The denoiser is an axial-attention transformer (the SceneDiffuser backbone) whose input is the element-concatenated, hidden-dimension-homogenized tensor, trained with the single v-prediction diffusion loss. Two sparse-tensor mechanisms carry the argument: during training, invalid entries are zeroed and the loss is masked so only the validity channel is supervised at invalid steps; during inference, soft clipping multiplies the denoised feature values by the predicted validity, so invalid entries decay toward zero instead of drifting out of distribution. That combination is what lets one model output a time-varying, sparse set of valid agents and lights, and it is the mechanism behind the paper's claimed ability to handle spawning, removal, occlusion, and traffic signals jointly.","core_discovery":"The central claim is that agent validity — whether an agent appears in the AV's detection output at a given timestep — can be treated as just another channel of the scene tensor and learned jointly with positions, sizes, and object types. Because validity is generated by the diffusion model itself rather than taken from the log, the model can emit sparse tensors without a prespecified sparsity structure: it learns when to insert a new agent, when to let an existing agent leave, and when an agent is occluded or disoccluded, by interpolating between valid and zero-invalid states during sampling. The same multi-tensor construction also carries a second tensor for traffic lights, with states and positions denoised alongside agents, so the entire trip is one inpainting problem. The paper reports state-of-the-art trip-level realism in 60-second rollouts, with substantially lower Jensen-Shannon divergence than IDM and SceneDiffuser for the distributions that depend directly on validity, while also matching logged traffic-light transition probabilities.","pith_inferences":["We would expect the sparse-tensor validity trick to transfer to other generative settings where outputs are naturally sparse, such as point-cloud completion with missing measurements or simultaneous generation of landmarks and objects; the paper does not test this.","A natural extension, not pursued in the paper, is to condition validity on the simulated ego's sensor footprint rather than on the logged AV's detection output, so that spawn and removal events remain physically meaningful when the simulated route diverges from the log.","The histogram-based Jensen-Shannon divergence protocol is the paper's own proposal; a reasonable next step for the field is to make such correspondence-free, sliding-window distributional metrics standard for long-horizon simulators."],"forward_implications":["Trip-level statistics such as travel time, pick-up and drop-off behavior, and long-horizon interactions with emergency vehicles can be generated from logged data alone, without hand-written insertion and removal rules.","Because the world model predicts traffic-light state transitions, violations of light rules can be measured in simulation rather than assumed from map priors.","The validity channel removes the hard limit that previously tied simulation duration to the length of the log: agents can be recycled into previously vacated slots, so arbitrarily many agents can appear over long horizons.","More frequent planner-world-model interaction lowers collision rates and improves speed realism, suggesting the model is most realistic when the AV and background agents continuously react to each other.","Rollout realism degrades as the horizon grows, especially for the timing of agent insertion, which indicates that autoregressive error accumulation, not map coverage, is the next bottleneck for trip-level simulation."],"supporting_citations":[{"why":"Supplies the scene-tensor diffusion formulation, the axial-attention denoiser backbone, and the SceneDiffuser baseline that SceneDiffuser++ extends and compares against.","marker":"[24]"},{"why":"Supplies the Waymo Open Motion Dataset; the map-extended WOMD-XLMap used for training and evaluation is built from it.","marker":"[12]"},{"why":"Supplies the Intelligent Driver Model baseline used for comparison and for routing behavior in the planner/world-model experiment matrix.","marker":"[49]"},{"why":"Provides the variance-preserving diffusion and v-prediction formulation used in the single denoising loss.","marker":"[20]"},{"why":"Provides the Jensen-Shannon divergence used to quantify distributional realism between simulated and logged metrics.","marker":"[28]"}],"fun_headline_variants":["City-scale traffic simulation from one diffusion world model","Single generative model simulates entire city trips, agents and signals","Diffusion learns when agents appear, vanish, or are occluded for city-scale trips","End-to-end world model predicts validity and traffic lights for point-A-to-B simulation","SceneDiffuser++: City simulator that generates sparse tensors and traffic states"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The validity channel is supervised using the logged AV's detection output, so 'valid' means visible to that particular sensor trajectory; in route-unconditioned rollouts the simulated AV drives elsewhere, and nothing in the training objective ties predicted validity to actual visibility from the simulated ego, so the spawned and removed agents may not be the ones the simulated AV could actually see.","fun_headline_variants_meta":{"raw":{"variants":["City-scale traffic simulation from one diffusion world model","Single generative model simulates entire city trips, agents and signals","Diffusion learns when agents appear, vanish, or are occluded for city-scale trips","End-to-end world model predicts validity and traffic lights for point-A-to-B simulation","SceneDiffuser++: City simulator that generates sparse tensors and traffic states"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000315,"raw_usage":{"total_tokens":1822,"prompt_tokens":1018,"completion_tokens":804,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":634,"completion_tokens_details":{"reasoning_tokens":709}},"tokens_in":634,"tokens_out":804,"duration_ms":7434,"temperature":1.0,"reasoning_tokens":709,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:14:25.105214+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replay 60-second SceneDiffuser++ rollouts with a fixed route-unconditioned planner, then ray-cast from the simulated ego pose against the map geometry to compute the physically visible agent set at each step; if the model's predicted validity (or its entering and exiting events) matches ray-cast visibility no better than a baseline that ignores map occlusion, the claim that the validity channel encodes occlusion and visibility would fail.","supporting_citations":[{"cited_title":"Scenediffuser: Efficient and controllable driving simulation initialization and rollout","cited_arxiv_id":null,"evidence_quote":"Supplies the scene-tensor diffusion formulation, the axial-attention denoiser backbone, and the SceneDiffuser baseline that SceneDiffuser++ extends and compares against."},{"cited_title":"Qi, Yin Zhou, Zoey Yang, Aur´elien Chouard, Pei Sun, Jiquan Ngiam, Vijay Vasudevan, Alexander McCauley, Jonathon Shlens, and Dragomir Anguelov","cited_arxiv_id":null,"evidence_quote":"Supplies the Waymo Open Motion Dataset; the map-extended WOMD-XLMap used for training and evaluation is built from it."},{"cited_title":"Con- gested traffic states in empirical observations and micro- scopic simulations","cited_arxiv_id":null,"evidence_quote":"Supplies the Intelligent Driver Model baseline used for comparison and for routing behavior in the planner/world-model experiment matrix."},{"cited_title":"sim- ple diffusion: End-to-end diffusion for high resolution im- ages","cited_arxiv_id":null,"evidence_quote":"Provides the variance-preserving diffusion and v-prediction formulation used in the single denoising loss."},{"cited_title":"Divergence measures based on the shannon en- tropy","cited_arxiv_id":null,"evidence_quote":"Provides the Jensen-Shannon divergence used to quantify distributional realism between simulated and logged metrics."}],"review_version":1}