{"id":"a4d47177-6775-4548-af9d-383bac21d5ae","arxiv_id":"2605.24037","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Mode-as-Sequence turns unordered multimodal trajectory sets into ordered sequences with explicit mode dependencies via recurrent or parallel decoding plus EMTA loss, yielding top rankings on Waymo motion prediction challenges.","lead":"The paper presents Mode-as-Sequence, a framework that converts sets of possible future trajectories into ordered sequences to better capture dependencies between modes in motion prediction. This approach aims to reduce mode collapse and improve confidence ranking under limited supervision, with potential use in autonomous vehicle planning.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Leaderboard 1st-place claims require isolating whether mode-to-mode dependency (not EMTA or other factors) drives the gains.","rationale":"The reader's weakest assumption correctly flags the risk that explicit mode ordering could introduce new failure modes under single-realization labels. My concern is prior and empirical: even if the modeling works as intended, the published 1st-place results do not yet demonstrate that this component (rather than EMTA or engineering) produced the ranking. The two concerns are adjacent but not identical; both reinforce the existing UNVERDICTED status rather than moving it to outright rejection.","tokens_in":1815,"tokens_out":360,"duration_ms":33211,"concrete_test":"Re-train the reported ModeSeq backbone on the 2024 LiDAR-free track using identical hyperparameters and EMTA but with independent (non-recurrent, non-masked) mode heads; compare minADE, minFDE, and ranking metrics to the full ModeSeq numbers. A drop larger than the reported margin over the next competitor would confirm the dependency modeling is load-bearing; comparable performance would indicate the claim over-attributes gains to the sequence formulation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim is the two 1st-place finishes on Waymo challenges. For this to substantiate the central thesis, the ordered sequence modeling (recurrent or masked-attention) plus EMTA must be shown to be the decisive ingredient rather than increased capacity, tuning, or the matching/ranking components alone. The abstract states \"consistent improvements\" and describes EMTA/MA-EMTA but supplies no ablation that removes only the causal dependency while keeping everything else fixed, leaving open the possibility that the headline numbers would hold under independent mode prediction.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes Mode-as-Sequence, a unified decoding framework that reformulates unordered multimodal motion prediction outputs as ordered mode sequences to explicitly capture mode-to-mode dependencies. It instantiates this via recurrent decoding (ModeSeq) and parallel masked self-attention (Parallel ModeSeq), introduces Early-Match-Take-All (EMTA) and MA-EMTA losses plus a ranking regularizer to mitigate mode collapse under single-realization supervision, and reports consistent gains in ranking metrics and best-of-K accuracy, culminating in first-place finishes on the 2024 Waymo LiDAR-free track and 2025 Interaction Prediction Challenge.","tokens_in":1963,"tokens_out":451,"duration_ms":21486,"significance":"If the central empirical claims are substantiated, the work offers a practical and scalable approach to improving mode diversity and confidence calibration in motion forecasting, directly evidenced by challenge leaderboard leadership; the parallel variant additionally addresses inference efficiency for large K and joint-scene settings.","major_comments":[{"comment":"Abstract (final paragraph) and method description: the headline 1st-place claims on Waymo challenges are presented as validation of the Mode-as-Sequence thesis, yet no ablation is described that holds EMTA/MA-EMTA, capacity, and ranking regularizer fixed while removing only the ordered mode-to-mode dependency (recurrent or masked-attention); without this isolation it remains possible that the reported gains are driven by the matching/ranking components rather than the sequential modeling itself.","section":"Abstract"},{"comment":"Abstract (paragraph 1) and results sections: the claim of 'consistent improvements ... across datasets, horizons, and object types' is stated without reference to error bars, statistical significance tests, or the number of random seeds; given the low-confidence soundness assessment arising from absent ablation tables, this weakens the ability to attribute gains specifically to the proposed dependency modeling.","section":"Abstract"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":"The two first-place claims are unusually strong selling points; the editor may wish to request the exact submission configurations and any post-challenge ablations that isolate the sequential component before final acceptance."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on the empirical validation of Mode-as-Sequence. We respond to each major comment below.","responses":[{"response":"We agree that a controlled ablation isolating the ordered mode-to-mode dependency (while holding EMTA/MA-EMTA, model capacity, and the ranking regularizer fixed) would more directly attribute gains to the sequential modeling. The manuscript compares ModeSeq/Parallel ModeSeq to non-sequential baselines, but these differ in multiple respects. We will add the requested ablation (e.g., independent per-mode decoding with identical losses and regularizer) in the revised version.","revision_made":"yes","referee_comment":"[Abstract] Abstract (final paragraph) and method description: the headline 1st-place claims on Waymo challenges are presented as validation of the Mode-as-Sequence thesis, yet no ablation is described that holds EMTA/MA-EMTA, capacity, and ranking regularizer fixed while removing only the ordered mode-to-mode dependency (recurrent or masked-attention); without this isolation it remains possible that the reported gains are driven by the matching/ranking components rather than the sequential modeling itself."},{"response":"We acknowledge that the absence of error bars, seed counts, and significance tests limits the strength of the consistency claims. Experiments were run across multiple seeds, but these details were omitted. We will revise the results sections and tables to report standard deviations, the number of random seeds, and any applicable statistical tests.","revision_made":"yes","referee_comment":"[Abstract] Abstract (paragraph 1) and results sections: the claim of 'consistent improvements ... across datasets, horizons, and object types' is stated without reference to error bars, statistical significance tests, or the number of random seeds; given the low-confidence soundness assessment arising from absent ablation tables, this weakens the ability to attribute gains specifically to the proposed dependency modeling."}],"tokens_in":1482,"tokens_out":419,"duration_ms":19521,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's core move is to treat the usual unordered set of predicted modes as an ordered sequence so that each new mode can be conditioned on the ones before it, either through recurrence or masked self-attention. They pair this with an Early-Match-Take-All loss that tries to match modes early and avoid collapse under the single-ground-truth supervision that motion forecasting always has. The parallel version removes the sequential bottleneck while keeping the same dependency structure. Those pieces got them first place in the 2024 LiDAR-free track and the 2025 Interaction Prediction Challenge on Waymo.\n\nThe empirical results are the strongest part: consistent gains on ranking metrics and best-of-K accuracy across datasets and object types, plus the two challenge wins. That is concrete evidence the approach is at least competitive in a crowded subfield.\n\nThe main gap is the missing ablation that keeps EMTA, capacity, and everything else fixed but removes only the mode-to-mode causal link. The abstract claims the dependency modeling is what drives diversity and calibrated ranking, yet the stress-test note correctly points out that no such controlled comparison is described. Without it, the headline numbers could be coming from the matching loss or from extra tuning rather than the sequence structure itself. The provided text also lacks error bars and detailed derivation of the loss, which keeps the soundness assessment provisional.\n\nThis is squarely for researchers working on multimodal trajectory prediction in autonomous driving. Anyone already running on Waymo or similar benchmarks will find the empirical numbers and the two instantiations worth looking at. It is coherent enough and grounded enough in external challenges to deserve a full referee process rather than a desk reject.","headline":"The sequence reframing plus EMTA loss produced real leaderboard wins on Waymo, but without an ablation isolating the causal dependency the central claim stays under-supported.","tokens_in":2477,"tokens_out":407,"would_cite":false,"duration_ms":22504,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Converting motion prediction modes into ordered sequences with explicit dependencies reduces collapse and improves ranking.","keywords":["multimodal motion prediction","mode collapse","sequence modeling","motion forecasting","autonomous driving","Waymo Open Dataset","recurrent decoding","masked attention"],"falsifier":"A controlled comparison on held-out scenes where the sequential models produce lower best-of-K accuracy or higher confidence inversion rates than a non-sequential multimodal baseline that uses the same backbone and matching loss.","tokens_in":2723,"feed_emoji":"🚗","tokens_out":703,"duration_ms":25076,"temperature":0.7,"pith_summary":"The paper claims that multimodal motion forecasting is under-supervised because each scene supplies only one realized future, which causes mode collapse into redundant hypotheses and unreliable confidence scores. It addresses this by reframing the unordered set of predicted modes as an ordered sequence so that mode-to-mode dependencies can be modeled directly. Two implementations follow: recurrent decoding in ModeSeq conditions each new mode on prior ones, while Parallel ModeSeq uses masked self-attention to keep the same causal structure but decode all modes at once. Early-Match-Take-All matching and a lightweight ranking regularizer further ensure the modes are representative and properly ordered under sparse labels. These changes produce measurable gains in both diversity and ranking metrics on large benchmarks and secure first-place results in recent Waymo challenges.","feed_headline":"Ordering modes as sequences improves motion prediction diversity and ranking","feed_subtitle":"Mode-as-Sequence modeling with recurrence or masked attention yields first-place results on Waymo LiDAR-free and interaction tracks.","key_machinery":"Mode-as-Sequence framework, which imposes sequential order on modes to enable explicit dependency modeling through recurrence or masked attention.","core_discovery":"Mode-as-Sequence is a unified decoding framework that translates an unordered mode set into an ordered mode sequence and explicitly models mode-to-mode dependency; ModeSeq realizes this via recurrent generation while Parallel ModeSeq uses masked mode-to-mode self-attention, both paired with Early-Match-Take-All matching and a ranking regularizer to learn calibrated modes from single-realization supervision.","pith_inferences":["The same sequence-ordering idea could be tested on other under-supervised multimodal tasks such as future video frame prediction or multi-hypothesis object detection.","If the causal dependency structure generalizes, it may offer a template for stabilizing diversity in other generative models trained with sparse supervision.","Large-scale deployment would require checking whether the added ordering constraints remain beneficial when scene complexity or sensor noise increases beyond the Waymo distribution."],"forward_implications":["Recurrent and parallel variants both improve ranking-oriented metrics and best-of-K accuracy across datasets, prediction horizons, and object types.","Parallel ModeSeq removes the autoregressive bottleneck, enabling efficient inference for large numbers of modes and joint-scene prediction.","MA-EMTA extends the matching strategy to multi-agent scenes while preserving the same dependency modeling.","The ranking regularizer directly reduces confidence inversions under the single-label regime."],"fun_headline_variants":["Mode-as-Sequence translates modes into ordered sequences for prediction","Recurrent ModeSeq models mode-to-mode dependencies in motion forecasting","Masked attention in Parallel ModeSeq enables efficient mode decoding","Early-Match-Take-All learns representative modes from single realizations"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That imposing ordered mode-to-mode dependencies through recurrence or masked attention, together with Early-Match-Take-All matching, will increase diversity and ranking accuracy without creating new failure modes when only one future is observed per scene.","fun_headline_variants_meta":{"raw":{"variants":["Mode-as-Sequence translates modes into ordered sequences for prediction","Recurrent ModeSeq models mode-to-mode dependencies in motion forecasting","Masked attention in Parallel ModeSeq enables efficient mode decoding","Early-Match-Take-All learns representative modes from single realizations"]},"model":"grok-4.3","cost_usd":0.006496,"raw_usage":{"total_tokens":3076,"prompt_tokens":740,"num_sources_used":0,"completion_tokens":66,"cost_in_usd_ticks":64962000,"prompt_tokens_details":{"text_tokens":740,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2270,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":740,"tokens_out":66,"duration_ms":18748,"temperature":1.0,"reasoning_tokens":2270,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T17:15:13.527640+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A controlled comparison on held-out scenes where the sequential models produce lower best-of-K accuracy or higher confidence inversion rates than a non-sequential multimodal baseline that uses the same backbone and matching loss.","supporting_citations":[],"review_version":1}