{"id":"beb32f81-aa32-4b4a-8b46-f996f1ddad8b","arxiv_id":"2502.08664","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey that organizes motion forecasting research into scenario-based and perception-based pipelines and into supervised and self-supervised learning, without introducing new results.","lead":"This paper surveys machine-learning methods that predict where vehicles, pedestrians, and other road users will move in the next few seconds, separating methods that use structured maps and trajectories from methods that learn directly from raw sensor data. A general reader could use it as a map of current motion forecasting research for autonomous vehicles.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 5.2.5 places supervised autoregressive language-modeling predictors (MotionLM, STR, AMP) under 'Self-Supervised Learning,' contradicting the paper's own pretrain-finetune definition in Section 5.2 and undermining the central supervised/self-supervised classification.","rationale":"The reader's weakest assumption targeted the scenario/perception axis; I focus on the supervised/self-supervised axis because the paper's own Section 5.2.5 gives a direct counterexample to that axis. This is more decisive than the scenario/perception boundary issue: even if one accepts the scenario/perception distinction, the learning-paradigm classification that the abstract foregrounds is violated by the methods used as examples. The paper's language-modeling subsection sits under the Self-Supervised Learning heading but contains methods whose objective is standard supervised next-token prediction on ground-truth trajectories. The absence of a pretrain/finetune stage for MotionLM, STR, and AMP distinguishes them from Traj-MAE, Forecast-MAE, POP, SEPT, and other methods in the same section; grouping them together conflates architecture with training paradigm. A concrete reclassification test would settle whether the claimed two-category structure can be made consistent. Secondary factual problems noted by the reader remain: Table 1 lists Waymo length as 9s although Section 4.1 says each scene is 20s, and the MR definition in Section 4.2 describes a count of scenarios rather than the standard miss rate; these also need correction, but the taxonomy mismatch is the load-bearing issue because it concerns the survey's core organizational contribution. Verdict remains CONDITIONAL: the classification can likely be repaired by moving language-modeling methods to the supervised section and stating explicitly that the survey covers self-supervised representation learning only for scenario-based inputs, but as written the central claim is not fully sound.","tokens_in":19020,"tokens_out":10053,"duration_ms":88229,"concrete_test":"Construct a probe table for all methods in Sections 5.2.1 through 5.2.5 with three columns: (1) does training use a separate unlabeled-data pretext stage? (2) is the primary objective maximum-likelihood on ground-truth future trajectories? (3) does the paper's Figure 7 pretrain/finetune flow apply? For MotionLM, STR, and AMP, the answers should be (no, yes, no) by the paper's own descriptions. If so, Section 5.2.5 is misclassified and the survey's central supervised/self-supervised partition is not internally consistent. Cross-check by moving these methods into Section 5.1 and confirming that the remaining Section 5.2 consists only of pretext-task methods; if the taxonomy then becomes clean, the paper needs to either reclassify the language-modeling methods or explicitly redefine what 'self-supervised' means in the abstract.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The survey's central classification into supervised and self-supervised learning is internally inconsistent. Section 5.2 defines self-supervised learning as representation learning from large-scale unlabeled data for downstream tasks, with the pretrain/finetune pipeline shown in Figure 7. Section 5.2.5 then groups MotionLM (Seff et al., 2023), STR (Sun et al., 2024), and AMP (Jia et al., 2024) as 'language modeling methods.' These models are trained with a maximum-likelihood objective on ground-truth future trajectory tokens; MotionLM is explicitly 'trained using a maximum likelihood objective over multi-agent action sequences.' They do not have a separate unlabeled-data pretraining phase or a finetuning phase in which a pretrained encoder is loaded, which is the criterion used for every other entry in Section 5.2. If language-modeling trajectory predictors count as self-supervised merely because they predict the next token, then the binary split collapses: most standard supervised trajectory predictors also predict future states from past observations. The paper therefore does not actually deliver the 'two main categories' promised in the abstract; it mixes training paradigm (supervised vs self-supervised) with model architecture (autoregressive vs non-autoregressive). This is not an external disagreement about definitions; it is an internal mismatch between Section 5.2's stated definition and the methods listed under it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a survey of motion forecasting for autonomous vehicles. It proposes a formal problem formulation, contrasts scenario-based and perception-based forecasting, reviews public datasets and evaluation metrics, and organizes recent deep-learning methods into supervised and self-supervised learning architectures. The survey covers representative work in temporal-spatial encoding, trajectory decoding, masked autoencoders, data augmentation, and language-modeling approaches, and it closes with future research directions.","tokens_in":19283,"tokens_out":3084,"duration_ms":30566,"significance":"A well-organized survey of this rapidly moving area is potentially valuable as a reference, and the paper does bring together a broad set of methods, datasets, and metrics, including recent self-supervised and language-modeling lines. The proposed scenario/perception dichotomy is a useful organizing axis, and the detailed enumeration of encoder and decoder designs gives the reader a compact entry point into the literature. However, the paper's central claim to classify methods into supervised and self-supervised learning is not internally consistent, and several technical definitions in the metrics section are imprecise. These issues must be corrected before the survey can serve as a reliable reference.","major_comments":[{"comment":"The subsection 'Language Modeling Methods' is placed under 'Self-Supervised Learning-based Architecture' (Section 5.2), but the paper's own definition of self-supervised learning in Section 5.2 is representation learning from large-scale unlabeled data via a pretrain/finetune pipeline (Figure 7). MotionLM, STR, and AMP are autoregressive sequence models trained with maximum-likelihood objectives on ground-truth trajectory tokens; they do not have an unlabeled pretraining phase or a finetuning phase in which pretrained encoders are loaded. Grouping them under SSL contradicts the stated definition and collapses the supervised/self-supervised dichotomy promised in the abstract. The authors should either move these methods to the supervised-learning section, or revise the definition and explain how next-token prediction qualifies as self-supervised under that definition.","section":"Section 5.2.5"},{"comment":"The Waymo Open Motion Dataset is described in Section 4.1 as containing scenes that are '20 seconds long and sampled at 10 Hz,' but Table 1 lists the Waymo duration as '9s.' These two statements are inconsistent. Since Table 1 is intended as a compact reference for dataset properties, the authors should reconcile the discrepancy or state explicitly whether '9s' refers to a different quantity (for example, the forecasting horizon).","section":"Section 4.1 and Table 1"},{"comment":"Miss Rate (MR) is defined as 'the number of scenarios in which all predicted trajectories deviate by more than 2.0 meters from the ground truth, as measured by endpoint error.' As written, this is a count, not a rate. In the motion-forecasting literature, MR is conventionally a fraction of scenarios, and a count would not be comparable across datasets of different sizes. The definition should be corrected to a normalized rate, or the metric should be renamed if a raw count is intended.","section":"Section 4.2, Miss Rate"},{"comment":"In the task-level metric, the function f(a, |∇ˆs(t:T)c|) is not defined, and the notation |∇ˆs(t:T)c| is unclear without specifying over which variables the gradient is taken and how the norm is computed. Additionally, the sentence introducing the metric says s_a^{(t:T)} and \\hat{s}_a^{(t:T)} are the 'predicted positions' and 'ground truth positions,' respectively, while the formula places them in the opposite order inside Metric(·). The equation and surrounding text should be revised so that the weighting scheme, the gradient notation, and the argument order are unambiguous and reproducible.","section":"Section 4.2, Eq. (16)"}],"minor_comments":[{"comment":"The vertical-axis label 'Y ear' contains a typo and should read 'Year.'","section":"Figure 1"},{"comment":"The summation limits and indices in minADE and minFDE are not motivated: the sum runs from t = T_obs to T_pred - 1, but the relationship between T_obs, T_pred, and the forecast horizon is not stated in the text. A brief explanation of these indices would help readers apply the formulas.","section":"Eq. (13) and Eq. (14)"},{"comment":"The paragraph on the Interaction dataset reports 'over 41,000' trajectories and 'more than 990 minutes' of driving, but the table lists no length or scene count for Interaction. Adding these values to Table 1 would make the comparison complete.","section":"Section 4.1"},{"comment":"Several citations in the text, such as 'Liu et al (2021)' in Table 1, do not have a matching full reference entry in the bibliography; the authors should check that all cited works appear in the reference list with complete author information.","section":"References"},{"comment":"The challenge titled 'Fusion of road information' is described at a high level but does not cite specific methods that address HDMap alignment and fusion; a pointer to one or two representative approaches would make the discussion more actionable.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe survey is a mixed bag. It gives a broad, readable map of motion forecasting work, and the scenario-based / perception-based split is a sensible organizing axis for a newcomer. The reference list is current and the breakdown of SSL methods (MAE, augmentation, pretext tasks) is a useful quick reference.\n\nBut the paper's central promise doesn't hold up. The abstract says it classifies recent research into supervised and self-supervised learning, and Section 5.2 defines self-supervised learning as representation learning from large-scale unlabeled data with a pretrain/finetune pipeline. Then Section 5.2.5 puts MotionLM, STR, and AMP under self-supervised learning. These are autoregressive models trained with maximum likelihood on ground-truth future tokens; they have no unlabeled-data pretraining phase. That's not a definitional quibble—it's an internal contradiction that collapses the main classification. If next-token prediction counts as self-supervised, then most supervised trajectory predictors qualify too.\n\nThere are also smaller factual issues: Miss Rate is defined as a count instead of a rate (it should be a proportion of scenarios); Table 1 says Waymo is 9s while Section 4.1 says 20s; and Eq. (16) has undefined notation. Figure 1's publication counts have no stated methodology, so they can't be verified.\n\nOn the positive side, the survey covers a wide range of methods and datasets, and the descriptions are largely consistent with the original papers. The perception-based section, including joint detection-prediction models like ViP3D and UniAD, is a nice touch.\n\nWho is this for? A newcomer who wants a high-level map and pointers to papers. But in its current form it can't be trusted as a reference for definitions or numbers, and the classification error is systemic, not cosmetic.\n\nI'd send it to peer review because the topic deserves a good survey and the fixes are tractable, but the authors would need to either rework the supervised/self-supervised axis or explicitly reframe Section 5.2.5 as an architectural category. The metric and table errors need correcting too.","headline":"A useful broad survey of motion forecasting that is undermined by an internally inconsistent supervised/self-supervised classification and several factual errors.","tokens_in":19788,"tokens_out":4401,"would_cite":false,"duration_ms":35419,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This survey proposes a formal problem formulation for autonomous-vehicle motion forecasting and organizes recent work along two axes: scenario-based versus perception-based input, and supervised versus self-supervised learning.","keywords":["motion forecasting","autonomous driving","trajectory prediction","scenario-based","perception-based","supervised learning","self-supervised learning","evaluation metrics"],"falsifier":"A concrete check would be to count, across a recent conference's motion-forecasting papers, how many methods the survey's two axes place without an arbitrary tie-breaker; if a large fraction (say, more than a third) must be assigned to both input categories or to both training paradigms, the proposed partition fails.","tokens_in":18822,"feed_emoji":"🚗","tokens_out":5794,"duration_ms":47218,"temperature":0.7,"pith_summary":"This survey tries to give autonomous-vehicle motion forecasting a common vocabulary and a workable map. It defines target agents, ego and surrounding agents, formalizes scenario-based input as past agent states plus HD maps, and formalizes perception-based input as raw sensor data mapped directly to future trajectories. It then sorts recent deep-learning methods by training paradigm: supervised learning, which dominates, and self-supervised learning, which is emerging. A sympathetic reader would care because a stable formulation and shared datasets and metrics are what let separate results be compared and progress measured.","feed_headline":"Survey formalizes AV motion forecasting and maps its methods","feed_subtitle":"It sorts approaches by input type and training paradigm, and lists the standard datasets and metrics.","key_machinery":"The load-bearing object is the formal problem formulation itself: scenario-based input is written as $T_{obs} = \\{X_{TAs}, HDMap, X_{SAs}, Others\\}$ with predicted trajectory $\\hat{X}_{TA} = \\{(x_{ta,t+1}, y_{ta,t+1}), ..., (x_{ta,t+T}, y_{ta,t+T})\\}$, and perception-based forecasting is written as a learned map $f: Z_t \\rightarrow X_{t+T}$ from raw sensor data to future states. The taxonomy axes carry the argument: scenario-based versus perception-based input, and supervised versus self-supervised training. The pipeline diagrams, an encoder-decoder structure for supervised learning and a pretrain-finetune structure for self-supervised learning, are the organizational device that lets every surveyed method be placed.","core_discovery":"The paper's central claim is that motion forecasting can be stated as a clean mathematical problem with two standard pipelines. In scenario-based forecasting, the input is the historical states of target and surrounding agents plus an HD map, and the output is a set of future positions for each target agent over a prediction horizon; in perception-based forecasting, a learned function maps raw LiDAR, camera, and radar data directly to future trajectories. Recent methods then split into supervised-learning architectures, built from rasterized, graph, or attention encoders followed by anchor-conditioned or anchor-free decoders, and self-supervised-learning architectures, which pretrain encoders with contrastive learning, masked autoencoding, auxiliary tasks, or language-model-style token prediction before fine-tuning. The survey also organizes evaluation into geometry-level, probabilistic-level, and task-level metrics, and lines up the main public datasets behind them.","pith_inferences":["Editorial inference: a natural extension the paper leaves implicit is a third axis of input representation granularity (rasterized, vectorized, BEV, raw), since several surveyed methods differ more in representation than in training paradigm.","Editorial inference: the taxonomy predicts that self-supervised pretraining will become the default initialization for supervised forecasting models, a prediction that is testable by tracking whether new forecasting papers report gains from pretrained encoders.","Editorial inference: the formal distinction between marginal prediction $p(s_1)p(s_2)...p(s_n)$ and joint prediction $p(s_1,s_2,...,s_n)$ could be pushed further, because a joint model that factorizes poorly in practice would sit at the boundary and suggest a separate axis for interaction modeling."],"forward_implications":["A common problem formulation means results from scenario-based and perception-based papers can be described in the same terms, making cross-paper comparison less ambiguous.","The supervised/self-supervised split gives newcomers a direct route into the field: supervised methods supply the baseline architecture, while self-supervised methods supply the pretraining toolbox.","The three-level metric stack (geometry, probability, task) makes explicit that a forecast that is accurate in meters may still be poor for downstream planning, and vice versa.","Placing joint detection-and-prediction models inside the supervised section implies that perception-based forecasting is still understood primarily through its training objective rather than through a separate architectural theory."],"supporting_citations":[{"why":"Provides the contrast case of a vision-based prediction survey, which the paper excludes by restricting scenario-based inputs to structured agent states and HD maps.","marker":"Teeti et al (2022)"},{"why":"Supplies the Argoverse dataset and the ADE and FDE geometric metrics that anchor the evaluation section.","marker":"Chang et al (2019)"},{"why":"Supplies Argoverse 2, the 250,000-scenario dataset that defines the current scale for scenario-based forecasting benchmarks.","marker":"Wilson et al (2023)"},{"why":"Supplies the Waymo Open Motion Dataset, one of the core benchmark datasets for interactive motion forecasting.","marker":"Ettinger et al (2021)"},{"why":"Supplies the Interaction dataset, the reference benchmark for interactive driving scenarios such as intersections and roundabouts.","marker":"Zhan et al (2019)"},{"why":"Supplies the nuScenes multimodal dataset, the basis for perception-based forecasting work with varied weather and city conditions.","marker":"Caesar et al (2020)"},{"why":"Supplies the Trajectron++ model and the negative log likelihood probabilistic metric used for evaluating multimodal output distributions.","marker":"Salzmann et al (2020)"},{"why":"Supplies the task-level planning-informed metrics that connect forecasting quality to downstream planning performance.","marker":"Ivanovic and Pavone (2021)"},{"why":"Supplies the FaF model, the early perception-based joint detection and prediction approach that grounds the perception-based category.","marker":"Luo et al (2018)"},{"why":"Supplies Vectornet, the graph-based encoder that anchors the graph-encoding branch and also provides an early self-supervised auxiliary task.","marker":"Gao et al (2020)"}],"fun_headline_variants":["Survey formalizes AV motion forecasting problems","AV forecasting survey: two pipelines, two paradigms","Motion forecasting taxonomy: scenario vs perception","Survey splits AV forecasting by data and training","Formal problem, two pipelines, many methods"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The survey's organization depends on the assumption that every method falls cleanly into one of two input types and one of two training paradigms, rather than straddling both.","fun_headline_variants_meta":{"raw":{"variants":["Survey formalizes AV motion forecasting problems","AV forecasting survey: two pipelines, two paradigms","Motion forecasting taxonomy: scenario vs perception","Survey splits AV forecasting by data and training","Formal problem, two pipelines, many methods"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1189,"prompt_tokens":855,"completion_tokens":334,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":471,"completion_tokens_details":{"reasoning_tokens":268}},"tokens_in":471,"tokens_out":334,"duration_ms":3810,"temperature":1.0,"reasoning_tokens":268,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T15:50:32.492420+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check would be to count, across a recent conference's motion-forecasting papers, how many methods the survey's two axes place without an arbitrary tie-breaker; if a large fraction (say, more than a third) must be assigned to both input categories or to both training paradigms, the proposed partition fails.","supporting_citations":[{"cited_title":"In: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, Lud De Raedt, Ed, pp 5630--5637","cited_arxiv_id":null,"evidence_quote":"Provides the contrast case of a vision-based prediction survey, which the paper excludes by restricting scenario-based inputs to structured agent states and HD maps."},{"cited_title":"In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8748--8757","cited_arxiv_id":null,"evidence_quote":"Supplies the Argoverse dataset and the ADE and FDE geometric metrics that anchor the evaluation section."},{"cited_title":"arXiv preprint arXiv:230100493","cited_arxiv_id":null,"evidence_quote":"Supplies Argoverse 2, the 250,000-scenario dataset that defines the current scale for scenario-based forecasting benchmarks."},{"cited_title":"In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 9710--9719","cited_arxiv_id":null,"evidence_quote":"Supplies the Waymo Open Motion Dataset, one of the core benchmark datasets for interactive motion forecasting."},{"cited_title":"In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11621--11631","cited_arxiv_id":null,"evidence_quote":"Supplies the nuScenes multimodal dataset, the basis for perception-based forecasting work with varied weather and city conditions."},{"cited_title":"In: Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XVIII 16, Springer, pp 683--700","cited_arxiv_id":null,"evidence_quote":"Supplies the Trajectron++ model and the negative log likelihood probabilistic metric used for evaluating multimodal output distributions."},{"cited_title":"arXiv preprint arXiv:210710297","cited_arxiv_id":null,"evidence_quote":"Supplies the task-level planning-informed metrics that connect forecasting quality to downstream planning performance."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11525--11533","cited_arxiv_id":null,"evidence_quote":"Supplies Vectornet, the graph-based encoder that anchors the graph-encoding branch and also provides an early self-supervised auxiliary task."}],"review_version":1}