{"id":"5af4a9ba-2544-4172-8a77-48e1d9476da6","arxiv_id":"2501.04815","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PerReg+ combines self-distillation, masked reconstruction, register queries, and prompt tuning in a Perceiver-based trajectory predictor, reporting improved accuracy on nuScenes, Argoverse 2, and Waymo.","lead":"This paper describes a new computer-vision system that predicts where nearby cars will go by learning both broad scene details and fine-grained motion patterns, then fine-tuning itself with small adjustable prompts on new data. The authors report that their system beats several existing predictors on three driving benchmarks and transfers better to a new city.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim depends on unverified protocol-equivalent baselines (Sec. 7) and is contradicted by Table 1 on Argoverse 2 single-dataset; both need resolution.","rationale":"The reader's weakest assumption—protocol-equivalent baselines—is the most load-bearing because the abstract's SOTA claim is an empirical statement; no amount of architectural novelty can rescue it if the numbers are not comparable. The paper provides independent support for the architecture's components (e.g., a principled combination of self-distillation and masked reconstruction, register queries, prompt tuning), and the ablations are suggestive, but the lack of code, seeds, and protocol-matched baselines makes the headline claim unverifiable. I agree with the reader's CONDITIONAL verdict: the paper could be acceptable if the SOTA claim is corrected (e.g., to 'competitive on several metrics' or explicitly multi-dataset) and code with protocol-matched baselines is released. The internal contradiction on Argoverse 2 strengthens the concern: it shows the claim as written is not supported even by the paper's own data. I do not see a more fundamental issue in the method itself; the self-distillation objective is plausible and the KoLeo regularizer mitigates collapse, and the prompt-selection mechanism, while underspecified, is not the basis for the central claim.","tokens_in":14685,"tokens_out":9056,"duration_ms":84045,"concrete_test":"Rerun all baselines (MTR, AutoBot, Forecast-MAE) inside the same UniTraj codebase with the authors' exact configuration (100m map radius, vehicle-only samples, 2s history/6s future, 6 modes, same metrics code) and report mean/std over at least 3 seeds. If any baseline B-FDE shifts by more than 0.05 on nuScenes, Argoverse 2, or WOMD, the SOTA claim fails. Additionally, rerun Forecast-MAE+ and PerReg+ on Argoverse 2 single-dataset with the same metric code to verify the 2.05 vs 2.07 gap; if PerReg+ does not beat Forecast-MAE+, the abstract's SOTA claim must be qualified to the multi-dataset setting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim—SOTA on nuScenes, Argoverse 2, and WOMD—requires that the baseline numbers in Table 1 are directly comparable to PerReg+ under identical data processing, map range, sampling, and metric code. Section 7 states that MTR and AutoBot results are 'sourced directly from the UniTraj paper' and Forecast-MAE was 'adapted' by the authors. If the UniTraj baselines used a different map range (e.g., 150m instead of the 100m radius used here), different agent filtering (e.g., including non-vehicles), or a different number of modes/metric implementation, the reported improvements could be artifacts. No error bars, seeds, or re-runs are provided. Moreover, even accepting the authors' protocol, Table 1 shows that on Argoverse 2 single-dataset training Forecast-MAE+ achieves B-FDE 2.05 vs PerReg+ 2.07, so PerReg+ is not SOTA on that setting. The abstract's unqualified 'sets a new state-of-the-art' is therefore either false or relies on an unspecified multi-dataset interpretation, which itself depends on the same comparability assumption. This is load-bearing because the entire contribution is framed around empirical superiority; if comparability fails, the central claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PerReg+, a trajectory prediction model built on the Perceiver IO architecture. It combines self-distillation and masked reconstruction for representation learning, adds register queries to the decoder for multimodal prediction, and uses prompt tuning during fine-tuning. Experiments are conducted on nuScenes, Argoverse 2, and WOMD within the UniTraj framework, comparing single-dataset and multi-dataset training, and including out-of-domain transfer from WOMD to nuScenes. The abstract claims state-of-the-art results on all three benchmarks and reports a 6.8% pretraining improvement and an 11.8% cross-domain B-FDE reduction.","tokens_in":19,"tokens_out":4812,"duration_ms":104764,"significance":"If the empirical results held, the paper would offer a useful recipe for pretraining and fine-tuning trajectory prediction models that avoids clustering/NMS and adapts efficiently to new datasets. The ablations, scalability experiments, and cross-domain evaluation are valuable. However, the headline SOTA claim is not supported by the paper's own Table 1 for Argoverse 2, and the abstract's pretraining improvement figure (6.8%) is inconsistent with Section 4.2 and the numbers in Table 1. The protocol under which baselines were evaluated is only partially described, and no error bars are provided. Because the central claims are empirical, these issues need to be resolved before the results can be accepted as stated.","major_comments":[{"comment":"The abstract claims that PerReg+ 'sets a new state-of-the-art performance on nuScenes, Argoverse 2, and Waymo Open Motion Dataset (WOMD).' In Table 1, single-dataset training on Argoverse 2 shows Forecast-MAE+ achieving B-FDE 2.05, minADE 0.74, minFDE 1.43, and MR 0.19, while PerReg+ achieves 2.07, 0.77, 1.46, and 0.21. PerReg+ is worse on every metric. In multi-dataset training, MTR achieves B-FDE 1.99 versus PerReg+'s 2.02, and other baselines also beat PerReg+ on some metrics. The SOTA claim for Argoverse 2 is therefore contradicted by the paper's own table. Please either correct the claim or re-evaluate to identify a setting in which PerReg+ is actually state-of-the-art.","section":"Table 1, Argoverse 2 rows"},{"comment":"The abstract states that pretraining 'reduces the error by 6.8% on smaller datasets.' Section 4.2 reports improvements of 11% on nuScenes, 13% on Argoverse 2, and 2.4% on WOMD. From Table 1, the B-FDE reductions for PerReg to PerReg+ are (3.06 to 2.62) = 14.4%, (2.38 to 2.07) = 13.0%, and (2.10 to 2.05) = 2.4%. None of these is 6.8%. The paper needs to reconcile the abstract, Section 4.2, and Table 1, and to specify exactly which comparison produces the 6.8% figure.","section":"Abstract vs. Section 4.2 and Table 1"},{"comment":"The SOTA claim depends on direct comparison with baselines, but Section 7 says MTR and AutoBot results are 'sourced directly from the UniTraj paper' and Forecast-MAE was 'adapted' to the UniTraj framework. Section 4.1 states the map range is 100m and only vehicle trajectories are used, yet it is not established that the UniTraj baselines were run under identical filtering, map range, prediction horizon (2s history / 6s future), and metric code. The paper also reports no error bars or multiple seeds. Please provide a detailed protocol-equivalence statement, or re-run the baselines under the same protocol, and restrict any SOTA claim to the settings that are actually controlled.","section":"Section 7 and Section 4.1"},{"comment":"The ablation text says Masked SD (+MSD) 'lowers B-FDE to 2.76', but Table 3 reports 2.64 for +MSD. Additionally, the caption of Table 3 describes '+SR' as 'self-distillation (SR)', while the text and table row use '+SR' to refer to segment-level reconstruction. These inconsistencies make it difficult to interpret which component produces which gain and should be corrected.","section":"Section 4.4 and Table 3"}],"minor_comments":[{"comment":"'Remarkable, our pretrained model...' should read 'Remarkably, our pretrained model...'.","section":"Abstract"},{"comment":"The model is called PerReg+ in the title and abstract, but Table 1 and Section 4.2 use 'PerReg (Ours)' with a '+' marker for pretrained variants. Please standardize the notation, for example by writing 'PerReg+' and 'PerReg (no pretraining)'.","section":"Throughout"},{"comment":"'A V2' is written with a space in several places; use 'Argoverse 2' or 'AV2' consistently.","section":"Section 4.2"},{"comment":"The description of the baseline as 'all decoder queries are used for prediction and aggregated via NMS' is unclear. Please clarify whether NMS is used in the baseline and how the final PerReg+ model eliminates the need for clustering and suppression, since this is one of the paper's stated contributions.","section":"Section 4.4"},{"comment":"Section 3.4 defines three losses (Ldistill, LGMM, Lrecon) in Equation (8), but the supplementary describes six losses for DWA. Please explain how the six losses map to the three terms in Equation (8), including the role of the KoLeo regularization term.","section":"Section 3.4 and supplementary Section 6.1"},{"comment":"The statement that 'only Forecast-PEFT keeps the pretrained decoder during finetuning' is a strong claim; please support it with a broader citation search or soften it to 'to the best of our knowledge' in a way that is clearly scoped.","section":"Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's evaluation is conducted inside UniTraj, a framework co-authored by members of the same group, and the SOTA claim is stated without qualification. This is not by itself improper, but the inconsistencies between the abstract, Section 4.2, and Table 1, together with the lack of baseline protocol details, mean the central empirical claim is not yet verifiable. I would like the editor to ensure the revised version either provides a fully controlled benchmark comparison or rewrites the claims to match the evidence in Table 1."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a believable engineering contribution, not a breakthrough. The assembly of Perceiver IO with dual-level self-distillation, masked reconstruction, segment-level reconstruction with query drop, register queries, and prompt-based fine-tuning hasn't been done before, as far as I know, and the ablations suggest each piece does something. Keeping the decoder during fine-tuning is a sensible move to preserve pretraining knowledge, and the cross-domain result (11.8% B-FDE reduction from pretraining) is the strongest evidence in the paper.\n\nThe soft spots are real. The abstract says 'sets a new state-of-the-art' on nuScenes, Argoverse 2, and WOMD, but Table 1 shows Forecast-MAE+ beating PerReg+ on Argoverse 2 single-dataset training (B-FDE 2.05 vs 2.07). If the claim is meant for multi-dataset training only, it needs to say so. Second, the baseline numbers are not protocol-verified: MTR and AutoBot are lifted from UniTraj, Forecast-MAE was adapted by the authors, and there are no error bars or re-runs. If map range, filtering, or metric code differ, the gains evaporate. That comparability issue is load-bearing for the SOTA claim. Third, the abstract's 6.8% improvement from pretraining does not match Section 4.2's 11% (nuScenes), 13% (Argoverse 2), and 2.4% (WOMD). Minor, but sloppy. No code or data is provided, so none of this is independently checkable.\n\nOn balance, I'd send it to review but require major revision: correct the SOTA claim, provide protocol-matched baselines or at least explicit confirmation of identical pre-processing, add error bars, and release code. The underlying method is plausible and the ablation is informative. A reader working on SSL for motion forecasting would get value from it. I wouldn't cite it as a SOTA reference until the numbers are cleaned up, but I'd engage with it.","headline":"A plausible SSL+Perceiver assembly for trajectory prediction whose unqualified SOTA claim is undercut by its own Argoverse 2 numbers and unverified baseline comparability.","tokens_in":15500,"tokens_out":2679,"would_cite":false,"duration_ms":24911,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Vehicle trajectory prediction can be made more generalizable by pretraining a single Perceiver-based architecture with self-distillation and masked reconstruction, then adapting it with prompt vectors; the authors report state-of-the-art…","keywords":["trajectory prediction","self-supervised learning","masked reconstruction","self-distillation","register queries","prompt tuning","multimodal prediction","autonomous driving"],"falsifier":"Re-run AutoBot, MTR, and Forecast-MAE inside the exact same code path and preprocessing—same 100-meter map radius, vehicle-only filtering, and same metric script—and compare B-FDE; if the rerun baselines match or beat PerReg+, the state-of-the-art claim collapses.","tokens_in":25,"feed_emoji":"🚗","tokens_out":5785,"duration_ms":114609,"temperature":0.7,"pith_summary":"Vehicle trajectory prediction models often overfit to one dataset and require expensive post-processing to pick among many candidate futures. This paper argues that a single Perceiver-based architecture can avoid those problems when it is pretrained with two complementary self-supervised tasks at once—self-distillation for global scene context and masked reconstruction for fine-grained detail—and then adapted with small prompt vectors instead of full fine-tuning. The resulting model, PerReg+, is claimed to set the best published numbers on nuScenes, Argoverse 2, and Waymo Open Motion Dataset, with pretraining reducing B-FDE by 6.8% on smaller datasets and by 11.8% in a cross-domain test. If the claim holds, trajectory prediction can be made more generalizable and cheaper to adapt without dataset-specific architectural surgery.","feed_headline":"One pretraining recipe tops three motion forecasting benchmarks","feed_subtitle":"Dual-level pretraining cuts cross-domain error by 11.8 percent, and small datasets gain 6.8 percent.","key_machinery":"The load-bearing object is the Perceiver IO encoder–decoder with a fixed-size latent array $Z_{\\mathrm{latent}} \\in \\mathbb{R}^{L \\times D}$, augmented by three decoder query types: mode queries for the $M$ predicted trajectories, register queries that act as structured memory, and segment reconstruction queries for agent histories, futures, and lane polylines. Pretraining couples two mechanisms: masked self-distillation, where a teacher encoder fed unmasked inputs (including future trajectories) is aligned with a student encoder fed masked inputs through a clustering head and cross-entropy loss, and masked reconstruction, where fine-grained masking with random query dropping forces the decoder to complete whole segments. Fine-tuning freezes the architecture and optimizes only a prompt pool $P=[p_1,\\dots,p_K]$, with the pretrained clustering head selecting the prompt for each scene.","core_discovery":"The paper's central claim is that a trajectory predictor does not need a bespoke architecture per dataset or clustering and suppression to handle multimodal futures. PerReg+ is a Perceiver IO model whose encoder is pretrained by having a teacher encoder process unmasked inputs, including future trajectories, while a student encoder processes heavily masked inputs; a cross-entropy loss aligns the two, and a decoder reconstructs entire past and future trajectory segments and lane segments from masked tokens. During fine-tuning the main architecture is frozen and only a small prompt pool, selected by the pretrained clustering head, is optimized. On the three benchmarks the pretrained variant is reported to outperform AutoBot, MTR, and Forecast-MAE on most metrics, and transferring from Waymo Open Motion Dataset to nuScenes reduces B-FDE by 11.8% relative to its non-pretrained counterpart.","pith_inferences":["Editorial extension: the same dual-level pretraining should improve data efficiency for other road users, such as pedestrians and cyclists, if the learned scene representation is genuinely general.","Editorial extension: prompt selection by clustering could be turned into a continual-learning mechanism—add a new prompt for a new city while freezing old prompts, then check whether old-domain accuracy degrades.","Editorial extension: the 11.8% cross-domain gain compares a model with pretraining against the same model without it; the practical question is whether the gain survives protocol-matched baselines and error bars."],"forward_implications":["Pretraining helps most where data is scarce: smaller datasets gain about 6.8% in B-FDE, so the recipe is a candidate for new cities or rare scenarios.","Retaining the pretrained decoder and optimizing only prompts makes adapting to a new dataset cheap in parameters and compute.","Register queries plus mode queries remove the need for trajectory clustering and non-maximum suppression, simplifying deployment.","Multi-dataset training improves or maintains accuracy on all three benchmarks, suggesting one model can serve several geographies.","Cross-domain transfer from a large dataset to a smaller one cuts B-FDE by 11.8% relative to no pretraining, supporting large-scale pretraining as a general route to generalization."],"supporting_citations":[{"why":"supplies the common evaluation protocol and the harness in which all models are trained and compared","marker":"[20]"},{"why":"the self-supervised masked-autoencoder baseline that PerReg+ must beat, adapted by the authors to the common protocol","marker":"[6]"},{"why":"the strong transformer baseline whose published tables are reused for comparison","marker":"[14]"},{"why":"the sequential-transformer baseline whose published tables are reused for comparison","marker":"[24]"},{"why":"introduces the self-distillation and teacher-student EMA scheme that the paper adapts to scene encoders","marker":"[19]"},{"why":"supplies the register-query idea that replaces clustering and suppression in the decoder","marker":"[21]"},{"why":"provides the prompt-tuning technique used for frozen-architecture adaptation","marker":"[43]"},{"why":"precedent for retaining the pretrained decoder during fine-tuning, which the paper builds on","marker":"[42]"},{"why":"the smallest benchmark, where pretraining gives the largest relative gain","marker":"[1]"},{"why":"the largest benchmark, used for pretraining and out-of-domain transfer","marker":"[3]"}],"fun_headline_variants":["Dual-level pretraining slashes trajectory error by 11.8%","PerReg+ tops nuScenes, Argoverse 2, and WOMD","Forget clustering: PerReg+ nails multimodal trajectory futures","Small datasets gain 6.8%; cross-domain error drops 11.8%"],"cache_read_input_tokens":17664,"weakest_assumption_plain":"The measured advantage depends on the baseline numbers being protocol-equivalent: two baselines are taken from the benchmark paper rather than rerun, the third was adapted by the authors, and no error bars are given, so a different map range, sample filter, or metric implementation could shrink or erase the reported gaps.","fun_headline_variants_meta":{"raw":{"variants":["Dual-level pretraining slashes trajectory error by 11.8%","PerReg+ tops nuScenes, Argoverse 2, and WOMD","Forget clustering: PerReg+ nails multimodal trajectory futures","Small datasets gain 6.8%; cross-domain error drops 11.8%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00101,"raw_usage":{"total_tokens":4273,"prompt_tokens":953,"completion_tokens":3320,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":3237}},"tokens_in":569,"tokens_out":3320,"duration_ms":24042,"temperature":1.0,"reasoning_tokens":3237,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:24:48.196834+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run AutoBot, MTR, and Forecast-MAE inside the exact same code path and preprocessing—same 100-meter map radius, vehicle-only filtering, and same metric script—and compare B-FDE; if the rerun baselines match or beat PerReg+, the state-of-the-art claim collapses.","supporting_citations":[{"cited_title":"Forecast-MAE: Self-supervised pre-training for motion forecasting with masked autoencoders","cited_arxiv_id":null,"evidence_quote":"the self-supervised masked-autoencoder baseline that PerReg+ must beat, adapted by the authors to the common protocol"},{"cited_title":"Motion transformer with global intention localization and local movement refinement","cited_arxiv_id":null,"evidence_quote":"the strong transformer baseline whose published tables are reused for comparison"},{"cited_title":"Latent variable sequential set transformers for joint multi-agent motion prediction","cited_arxiv_id":null,"evidence_quote":"the sequential-transformer baseline whose published tables are reused for comparison"},{"cited_title":"Bootstrap your own latent-a new approach to self-supervised learning","cited_arxiv_id":null,"evidence_quote":"introduces the self-distillation and teacher-student EMA scheme that the paper adapts to scene encoders"},{"cited_title":"Vision transformers need registers","cited_arxiv_id":null,"evidence_quote":"supplies the register-query idea that replaces clustering and suppression in the decoder"},{"cited_title":"Visual prompt tuning","cited_arxiv_id":null,"evidence_quote":"provides the prompt-tuning technique used for frozen-architecture adaptation"},{"cited_title":"nuscenes: A multimodal dataset for autonomous driving","cited_arxiv_id":null,"evidence_quote":"the smallest benchmark, where pretraining gives the largest relative gain"},{"cited_title":"Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset","cited_arxiv_id":null,"evidence_quote":"the largest benchmark, used for pretraining and out-of-domain transfer"}],"review_version":1}