{"id":"46f32d8b-67eb-4f5e-a6a6-dc272ef930ae","arxiv_id":"2411.19747","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Adding three auxiliary loss functions that supervise every predicted trajectory reduces off-road and wrong-direction predictions in autonomous vehicle motion forecasting with little accuracy loss.","lead":"This paper adds three extra training penalties to vehicle trajectory prediction models: one keeps predicted paths on the road, one aligns them with traffic direction, and one spreads out the predicted options. On two driving datasets the penalties cut off-road prediction errors by roughly half while keeping overall accuracy about the same.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The '+All' comparison is confounded by fine-tuning the baseline, and the evaluation metrics are the same formulas as the training losses, so the causal attribution of the reported gains is not established.","rationale":"The most load-bearing condition for the central claim is causal attribution: adding the three losses must be the reason the offroad, direction, and diversity metrics improve. In the current paper this condition is most threatened in the '+All' and '+Diversity' rows, which are fine-tuned from a pretrained baseline while the comparison baseline is not. Extra gradient steps alone can reduce training losses and often improve in-distribution metrics; without a fine-tuned control, the 47% and 37% offroad reductions in the abstract cannot be assigned to the new loss terms. The reader's 'same formula as loss' concern captures a related but distinct threat to external validity: even if the losses cause the metric changes, the metrics may not reflect real safety. My read is that the internal-validity confound is more decisive, because it questions whether the proposed losses matter at all. Nevertheless, the paper does include from-scratch experiments for Offroad and Direction losses and qualitative evidence, so the contribution may survive a matched control; hence the appropriate verdict remains conditional, meaning the reader's conditional verdict is unchanged.","tokens_in":10450,"tokens_out":9723,"duration_ms":81746,"concrete_test":"Run a matched control: initialize from the same pretrained baseline used for '+All', fine-tune for the same 10 epochs with the same optimizer, learning rate, and data, but with alpha=0 for all three auxiliary losses (i.e., baseline objective only). Evaluate this control on the same validation split using the same Offroad/Direction/Diversity/minADE metrics, over 3 seeds. If the control's Offroad metric is within roughly 10% of the '+All' value, or if its Diversity is similar to the '+Diversity' row, the reported gains are explained by fine-tuning rather than by the proposed losses. Also report means and standard deviations to determine statistical significance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline comparison in Table I does not isolate the effect of the proposed losses. Section IV-A states that models with Offroad and Direction losses are trained from scratch, but the Diversity loss is applied by fine-tuning the baseline for 10 epochs, and the combined '+All' model is also fine-tuned; the baseline row receives no equivalent fine-tuning. Thus the large Offroad reductions in '+All' rows (e.g., AutoBots Argoverse2: 0.30 to 0.19; Wayformer nuScenes: 2.73 to 1.67) and the Diversity gains (AutoBots nuScenes: 64 to 74) could be produced by additional optimization steps alone. The causal claim that the proposed losses improve all quality metrics is therefore not established by the reported experiments. Compounding this, the Offroad, Direction, and Diversity evaluation metrics are computed with the same formulas as the training losses (Eqs. 2-4), so improvements on those metrics are partly by construction, and the alpha used for each model is selected from the same trade-off curve (Fig. 4) rather than by a pre-registered protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes three auxiliary loss functions for multimodal vehicle trajectory prediction: Offroad Loss (Eq. 2), Direction Consistency Error (Eq. 3), and Mode Diversity Loss (Eq. 4). The losses are applied to all predicted modes rather than only the best mode, and are combined with the original training objective as Lfinal = Loriginal + alpha * Laux. The authors evaluate the approach with Wayformer and AutoBots on nuScenes and Argoverse 2, reporting that the auxiliary losses reduce the Offroad metric by about 47% on original scenes and 37% on attacked scenes, improve Direction and Diversity metrics, and broadly maintain minADE/minFDE. The paper also includes qualitative examples, an alpha-sweep study, and a robustness evaluation under the Scene Attack benchmark.","tokens_in":10710,"tokens_out":4308,"duration_ms":37585,"significance":"If the reported results are valid, the proposed losses offer a simple, model-agnostic way to improve road compliance and prediction diversity without architectural changes, which would be practically useful for autonomous driving. The paper provides a public code link and evaluates on two standard datasets with two baselines. However, the significance is substantially tempered by two issues: the evaluation metrics appear to be computed from the same formulas as the training losses, and the main comparison is confounded by an asymmetric fine-tuning protocol. The non-circular evidence, namely that minADE/minFDE are not severely degraded, is credible but reported without error bars or significance testing. Thus the headline safety and robustness claims need additional support before the paper's central contribution can be accepted as established.","major_comments":[{"comment":"The evaluation metrics called Offroad, Direction Error, and Diversity in Table I appear to be numerically identical to the training losses defined in Eqs. (2)-(4), yet their definitions are never explicitly given in Section IV-A. If they are the same quantities, then the reported improvements on these metrics are partly by construction: the optimizer is directly minimizing the evaluation yardstick. The manuscript should state the exact metric formulas and, more importantly, provide evidence from independent metrics that are not minimized during training, or explicitly frame the results as an optimization check rather than as independent safety evaluation.","section":"Section IV-A, Table I, Eqs. (2)-(4)"},{"comment":"The training protocol is asymmetric across rows of Table I. The Offroad and Direction losses are trained from scratch, while the Diversity loss is applied by fine-tuning the baseline for 10 epochs; the '+All' model is also fine-tuned. The baseline row receives no equivalent fine-tuning. Consequently, the gains in the '+Diversity' and '+All' rows (e.g., AutoBots nuScenes Diversity from 64 to 93, Wayformer Argoverse2 Direction from 4.24 to 2.54 with '+All') could be due to the additional optimization steps alone rather than to the proposed losses. The causal claim that the proposed losses improve all quality metrics requires a fair comparison with a baseline that is fine-tuned for the same number of epochs under the same schedule.","section":"Section IV-A, training strategy"},{"comment":"All quantitative results are reported as single numbers without error bars, number of seeds, or significance tests. In addition, the alpha hyperparameter is selected from the same trade-off curves shown in Fig. 4 that are used to report the final model, which introduces selection bias. The authors should either fix the alpha selection rule a priori, use a validation split for selection, or report variability across multiple training runs. Without this, it is not possible to assess whether differences such as Wayformer nuScenes minADE from 1.08 to 1.13 or AutoBots Argoverse2 minADE from 0.92 to 0.95 are meaningful.","section":"Section IV-B, Table I, Fig. 4"},{"comment":"The abstract claims an average 37% reduction in offroad errors on attacked scenes, but Table II does not directly reproduce this number for either '+Offroad' or '+All'. For '+All', the attacked-scene reductions range from 15.1% (Wayformer Argoverse2) to 34.1% (AutoBots nuScenes), and for '+Offroad' they range from 15.1% (Wayformer nuScenes) to 59.4% (Wayformer Argoverse2). The manuscript should specify exactly which rows are averaged, how the average is computed, and should report the per-scenario or per-attack-type variability. Similar precision is needed for the 47% original-scene claim.","section":"Section IV-E and Abstract"}],"minor_comments":[{"comment":"The heading angle gamma_i^t of a predicted trajectory point is used in Eq. (3), but the manuscript never defines how gamma_i^t is computed from the predicted positions. A short formula or reference would remove ambiguity.","section":"Section III-A, Eq. (3)"},{"comment":"The double sum in Eq. (4) includes the term i = j, whose pairwise distance is zero but whose indicator functions are still evaluated, and the notation applies 1(j) but not 1(i) in the summand. The formula should be cleaned up, for example by summing over i < j and explicitly applying both feasibility indicators.","section":"Section III-C.3, Eq. (4)"},{"comment":"The text says 'we introduce three novel measures' for evaluation, but no formulas are given for the Offroad, Direction Error, and Diversity metrics. The authors should state explicitly whether these metrics are the same functions as Eqs. (2)-(4) or whether they use different parameters, since this is central to interpreting Table I.","section":"Section IV-A"},{"comment":"The x-axis labels 'Offroad', 'Direction Consistency Error', and 'Diversity' should be clarified as metric names; if they are identical to the loss definitions, the figure should say so, and if not, the metric definitions should be given in the caption.","section":"Fig. 4"},{"comment":"The discussion of prior offroad losses in [25]-[29] correctly notes that they rely on rasterized masks, but the claim of novelty should be stated more carefully: the vectorized formulation is a useful engineering contribution, while the concept of an offroad loss is already present in the cited literature.","section":"Section II, Related Work"},{"comment":"The margin m in Eq. (2) is introduced but its value is never reported in the experiments. Since the margin affects both the loss and the resulting Offroad metric, the chosen values for m, m_d, and m_theta should be listed in the experimental setup.","section":"Section III-C.1, Eq. (2)"}],"recommendation":"major_revision","confidential_remarks":"The paper is straightforward and may be suitable for publication after revision, but the current evaluation does not support the strong benchmark and safety claims. The core problem is that the evaluation metrics are the same formulas as the training losses, and the '+All' comparison is confounded by fine-tuning. I would encourage the editor to ask for either a softening of the claims to 'we optimize the proposed metrics while maintaining accuracy' or, preferably, an evaluation with independent metrics, a fair fine-tuned baseline, and error bars. The authors should also be asked to explain the discrepancy between the abstract's 37% attacked-scene reduction and Table II."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a clearly written, useful paper on auxiliary losses for trajectory prediction, but the headline +All results are weaker than they look. The stress-test note is right: baseline rows do not get the extra fine-tuning that +Diversity and +All receive, so part of the gain may just be more optimization steps. And Table I's Offroad/Direction/Diversity metrics are literally the loss formulas, so reductions there are partly by construction. Still, there is a real contribution here.\n\nWhat is actually new: the vectorized off-road loss on GPU, the direction-consistency term that matches against all centerlines rather than only the nearest one, and the all-mode supervision scheme. The paper is honest about the small minADE/minFDE trade-offs, and the scene attack results at least test out-of-distribution robustness. Code is released, which is real evidence and makes the work reproducible.\n\nSoft spots, in proportion: the 'three novel loss functions' framing overstates the novelty. Off-road and diversity losses have direct cited antecedents; the novelty is the vectorized form and the combination. More importantly, the experimental protocol does not support the causal claim. There are no error bars or significance tests, alpha is selected from the same trade-off curves used to report results, and the fine-tuning asymmetry confounds the +Diversity and +All rows. The abstract's 'new benchmark' language is more than the evidence supports. That said, the central idea is not broken; the paper simply has not yet proven that the combined loss, rather than extra fine-tuning, drives the gains. A revision with equal-compute baselines, fixed alpha selection, and error bars would make this solid.\n\nWho this is for: anyone working on trajectory prediction or on auxiliary losses for safety-related metrics. The method is likely useful and the code makes it easy to try. I would send it to peer review, because the contribution is practical and the flaws are fixable. A good referee should push for a cleaner comparison and for independent safety metrics that are not identical to the training losses.","headline":"Useful auxiliary losses with a real confound: the +All gains are not cleanly attributable, and the headline metrics are the same formulas as the training losses.","tokens_in":11200,"tokens_out":2357,"would_cite":true,"duration_ms":22639,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Three auxiliary losses applied to every predicted trajectory reduce off-road errors by roughly half while preserving prediction accuracy.","keywords":["trajectory prediction","autonomous driving","auxiliary loss","off-road loss","direction consistency","diversity loss","scene compliance","multimodal prediction"],"falsifier":"Take a model trained with these losses and evaluate it on a scene set whose drivable-area polygons come from a different, independently produced map source than the one used in training; if the off-road metric is not markedly better than a baseline trained without the losses, then the improvement reflects overfitting to the training map annotations rather than generalizable road compliance.","tokens_in":1778,"feed_emoji":"🚗","tokens_out":2987,"duration_ms":90952,"temperature":0.7,"pith_summary":"Vehicle trajectory predictors are typically trained with a \"winner takes all\" objective that updates only the one trajectory closest to the ground truth, leaving the other modes unrefined and often producing predictions that leave the road, oppose traffic flow, or miss plausible maneuvers. This paper proposes three differentiable auxiliary losses—Offroad Loss, Direction Consistency Error, and Diversity Loss—that supervise every prediction mode, and shows on the nuScenes and Argoverse 2 datasets with Wayformer and AutoBots baselines that they cut the offroad metric by roughly half on original scenes and by about 37% on attacked scenes. The work also demonstrates that the losses can double as evaluation metrics for realism and diversity, and that combining all three yields a balanced improvement over all quality metrics with only minor changes in standard accuracy measures. A sympathetic reader would take the central claim to be that road-rule compliance and mode diversity are trainable properties separable from accuracy, and that any existing predictor can gain them without architectural changes.","feed_headline":"Three losses halve off-road errors in trajectory prediction","feed_subtitle":"Adding three loss terms to any predictor improves road compliance and diversity while barely changing accuracy.","key_machinery":"The central mechanism is a set of three fully differentiable auxiliary losses that encode geometric road knowledge and are applied uniformly across all prediction modes. Offroad Loss computes a signed distance to the union of drivable-area polygons and penalizes points beyond a margin, using ray casting for point-in-polygon tests and GPU-accelerated distance computations. Direction Consistency Error measures, for each predicted point, the minimum combined distance-and-yaw gap to any centerline point, allowing flexible matching in intersections where several centerlines with different directions coexist. Mode Diversity filters out off-road trajectories and then sums pairwise distances among the remaining feasible modes, spreading predictions without pushing them off-road. The losses are used as auxiliary terms in the total loss with a hyperparameter $\\alpha$ balancing their weight, and they also serve as evaluation metrics.","core_discovery":"The authors claim that augmenting the standard minADE objective with three auxiliary loss terms, applied to all $M$ predicted trajectories rather than only the best one, improves scene compliance and diversity across the board. Concretely, Offroad Loss uses a signed distance function to the drivable-area polygons, penalizing points outside a margin and pulling trajectories back onto the road; Direction Consistency Error matches each predicted point to the most suitable centerline by position and heading with margins, resolving the ambiguity of intersections and multi-lane roads; Diversity Loss sums pairwise distances among the feasible (on-road) modes, spreading the set to cover distinct plausible maneuvers. Across nuScenes and Argoverse 2, these losses reduce the offroad metric by roughly half in the best cases, improve directional alignment, and increase diversity, with small increases in minADE and minFDE that the authors frame as an acceptable safety-accuracy trade-off. When all three losses are combined, the models improve on every assessed quality metric relative to the baseline, and trained models also show lower offroad errors on scenes with synthetically introduced turns.","pith_inferences":["Because the evaluation metrics are computed with the same signed-distance and centerline formulas as the training losses, at least part of the reported improvement may reflect direct optimization of the metric itself; an independent road-compliance test with different map sources or human judgment would reveal how much of the gain is genuine behavioral change.","The losses could be extended to collision avoidance and kinematic feasibility, as the authors note, and the hand-tuned $\\alpha$ weights could be replaced by an adaptive weighting scheme based on the model's training state.","The diversity loss's feasibility filter could be reused in planning cost functions to discourage redundant trajectory options during downstream decision-making, not just during prediction training.","The vectorized GPU implementation suggests these losses are inexpensive enough to become a standard regularization in large-scale trajectory-prediction training, potentially shifting evaluation practice toward compliance-aware metrics."],"forward_implications":["Any existing trajectory prediction model can be improved in road compliance and mode diversity by adding these losses, without changing the network architecture.","The three losses provide a standardized, differentiable set of evaluation metrics, allowing future work to measure scene compliance independently of prediction accuracy.","Supervising all modes counters the winner-takes-all blind spot, so less frequent but plausible maneuvers such as continuing straight at an intersection are retained in the prediction set.","Models trained with the losses remain more road-compliant when the scene is perturbed by naturalistic turns, indicating improved robustness to out-of-distribution inputs.","The small accuracy cost, such as Wayformer's minADE rising from 1.08 to 1.13 on nuScenes with all losses, is presented as a deliberate trade-off favoring safety-critical road compliance."],"supporting_citations":[{"why":"Wayformer, one of the two baseline models the loss functions are added to and tested against.","marker":"[1]"},{"why":"AutoBots, the second baseline model used to show the losses work across different architectures.","marker":"[15]"},{"why":"The unified training framework that supplies the training setup, hyperparameters, and integration for adding auxiliary losses to the baselines.","marker":"[34]"},{"why":"nuScenes dataset, one of the two benchmarks on which all quantitative results are measured.","marker":"[35]"},{"why":"Argoverse 2 dataset, the larger benchmark that provides the second set of quantitative results.","marker":"[36]"},{"why":"The naturalistic attack benchmark that introduces turns to test robustness of models trained with the proposed losses.","marker":"[33]"},{"why":"Computational geometry algorithms for point-in-polygon tests and distance computations that underpin the signed-distance Offroad Loss.","marker":"[32]"},{"why":"The lane heading auxiliary loss that the Direction Consistency Error extends and aims to fix at intersections.","marker":"[30]"}],"fun_headline_variants":["Triple loss halves offroad violations in trajectory prediction","Road-rule losses cut offroad errors 47% without hurting accuracy","Three losses keep paths on-road, aligned, and diverse","Combined losses improve safety and diversity in trajectory forecasting"],"cache_read_input_tokens":13312,"weakest_assumption_plain":"The evaluation metrics that measure off-road, direction, and diversity are computed with the same signed-distance and centerline-matching formulas as the proposed training losses, so the reported safety improvements assume these metrics are faithful proxies for real driving compliance and that the drivable-area and centerline annotations are accurate.","fun_headline_variants_meta":{"raw":{"variants":["Triple loss halves offroad violations in trajectory prediction","Road-rule losses cut offroad errors 47% without hurting accuracy","Three losses keep paths on-road, aligned, and diverse","Combined losses improve safety and diversity in trajectory forecasting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000485,"raw_usage":{"total_tokens":2413,"prompt_tokens":982,"completion_tokens":1431,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":1363}},"tokens_in":598,"tokens_out":1431,"duration_ms":11835,"temperature":1.0,"reasoning_tokens":1363,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:52:00.094677+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a model trained with these losses and evaluate it on a scene set whose drivable-area polygons come from a different, independently produced map source than the one used in training; if the off-road metric is not markedly better than a baseline trained without the losses, then the improvement reflects overfitting to the training map annotations rather than generalizable road compliance.","supporting_citations":[{"cited_title":"nuscenes: A multimodal dataset for autonomous driving,","cited_arxiv_id":null,"evidence_quote":"nuScenes dataset, one of the two benchmarks on which all quantitative results are measured."},{"cited_title":"Wayformer: Motion forecasting via simple & efficient attention networks,","cited_arxiv_id":null,"evidence_quote":"Wayformer, one of the two baseline models the loss functions are added to and tested against."},{"cited_title":"Latent variable sequential set transformers for joint multi-agent motion prediction,","cited_arxiv_id":null,"evidence_quote":"AutoBots, the second baseline model used to show the losses work across different architectures."},{"cited_title":"Vehicle trajectory prediction works, but not everywhere,","cited_arxiv_id":null,"evidence_quote":"The naturalistic attack benchmark that introduces turns to test robustness of models trained with the proposed losses."},{"cited_title":"O’Rourke, Computational Geometry in C , 2nd ed","cited_arxiv_id":null,"evidence_quote":"Computational geometry algorithms for point-in-polygon tests and distance computations that underpin the signed-distance Offroad Loss."},{"cited_title":"Trajectory prediction in au- tonomous driving with a lane heading auxiliary loss,","cited_arxiv_id":null,"evidence_quote":"The lane heading auxiliary loss that the Direction Consistency Error extends and aims to fix at intersections."}],"review_version":1}