{"id":"b5db19f0-923d-482c-bc7c-cf200dd501ce","arxiv_id":"2507.02406","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Adapting SimPO preference optimization for multi-agent trajectory prediction lowers collision rates on Argoverse 2, Interaction, and Waymo Open Motion Dataset with minimal accuracy loss.","lead":"This paper applies preference optimization, a technique from large language models, to vehicle trajectory prediction, fine-tuning models so that predicted traffic scenes have fewer vehicle collisions. The method reduces collision rates across three driving datasets with only small losses in prediction accuracy and no extra computation at inference time.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported consistency gains are confounded with the order-based pairing heuristic used for both the SimPO target and the evaluation metric; it is unclear whether any real joint-consistency improvement survives an independent aggregation.","rationale":"Read in good faith, the paper does something concrete and mostly self-consistent: it defines an automatic preference cost, selects informative scenes, and fine-tunes with a Plackett-Luce/SimPO objective. The positive results across three datasets are plausible, and the ablation on the margin parameter supports that the loss, not just the learning rate, drives the effect. I do not see an internal mathematical contradiction in Eq. 12; the rank-scaled margin is a legitimate training objective, and the model is not required to represent a normalized joint distribution. The weakest point is external validity: because joint modes are constructed by pairing each agent's k-th trajectory, and because the same constructed modes define SCR, pSCR, and the likelihoods in the loss, the method can improve the headline numbers by rearranging pairings or by down-weighting a single agent's trajectory inside a colliding mode. The paper's own qualitative discussion and oversampling experiments admit that aggregation changes are the main mechanism of improvement. This does not make the method useless, but it means the central claim of improved scene consistency is currently demonstrated only for one particular, arbitrary aggregation. That is precisely the assumption the reader flagged, and the proposed test can settle whether the improvements reflect genuine joint consistency or an artifact of pairing. Since the finding would narrow rather than overturn the claim, the conditional verdict stands unchanged.","tokens_in":11822,"tokens_out":8824,"duration_ms":114096,"concrete_test":"Evaluate the fine-tuned QCNet checkpoint on Argoverse 2 under alternative joint-mode aggregations: (i) the paper's rank-order pairing, (ii) 100 random permutations of each agent's K predicted trajectories per scene, and (iii) the permutation per scene that minimizes total collision cost. Recompute SCR and pSCR for the baseline and SimPO-fine-tuned models under each aggregation. If the SimPO advantage over baseline is absent or much smaller under (ii) or (iii), the reported consistency improvement is an artifact of the rank-order pairing; if the advantage persists under all aggregations, the concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing flaw is that the consistency metric and the training objective are both defined through the same arbitrary rank-order pairing of per-agent trajectories (Section IV-C). For marginal models, the scene-level likelihood is computed by averaging per-agent logits across the agents whose k-th most likely trajectories are paired, and SCR and pSCR (Eqs. 13-14) count collisions inside exactly such paired modes. SimPO (Eq. 12) is then free to reduce collision rates by changing which per-agent modes are paired, or by suppressing a single agent's logit in a colliding mode, without making the underlying marginal predictions jointly plausible in any model-independent sense. The paper itself acknowledges this mechanism: Figure 2 and the K=15 experiments show that 'rearranging agent pairings' is the main effect. Consequently, the headline claim that preference optimization significantly improves scene consistency is not established outside the specific pairing used for both training and evaluation; a planner using a different joint-decoding scheme might see none of the reported benefit. This is a scope/validity concern rather than an internal contradiction, but it is exactly what conditional acceptance should require.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a fine-tuning method that adapts SimPO preference optimization to multi-agent trajectory prediction. Given a pretrained marginal or joint predictor, the method ranks predicted scene-level modes by a preference cost combining average FDE and a collision repeller term, extracts a subset of interactive training scenes, and fine-tunes the model with a Plackett-Luce variant of SimPO that uses rank-scaled margins. Experiments on Argoverse 2, Interaction, and Waymo Open Motion Dataset show reduced scene collision rate (SCR) and probability-weighted SCR (pSCR) for QCNet, FJMP variants, and BeTop, with only small degradation in MinJointFDE. The paper also studies the effect of oversampling modes and the sensitivity to the target reward margin γ and collision weight λ.","tokens_in":12102,"tokens_out":2371,"duration_ms":27766,"significance":"If the reported improvements are robust, this is a novel and practically relevant application of preference optimization to trajectory prediction: it is training-time only, model-agnostic, and does not add inference cost. The paper is careful to include ablations over γ and λ, to test both marginal and joint models, and to report accuracy trade-offs. The main limitation is that the scene-level likelihood aggregation and the evaluation metrics are both defined through the same order-based pairing of per-agent trajectories, which confounds the claimed consistency gains with the choice of aggregation heuristic.","major_comments":[{"comment":"The scene-level likelihoods used for both training and evaluation are constructed by pairing each agent's k-th most likely trajectory and averaging logits (Section IV-C). The SimPO loss in Eq. (12) and the metrics SCR/pSCR in Eqs. (13)-(14) are both computed on modes formed by this exact order-based pairing. Since the paper itself states that 'rearranging agent pairings to achieve consistent predictions is the main effect' (Section V-C), the reported SCR/pSCR improvements may be largely an artifact of the fixed pairing convention rather than evidence of model-independent joint consistency. To support the headline claim, the authors should evaluate with an independent aggregation procedure (e.g., random pairings, a different ranking-based pairing, or a jointly decoded set of modes) and show that the fine-tuned model still reduces collisions under that independent evaluation. Without such an experiment, the central claim is not established outside the specific pairing used for both training and evaluation.","section":"Section IV-C and V-B"},{"comment":"The rank-scaled margin term kγ is added inside the exponential in the Plackett-Luce loss. This changes the effective reward from β log πθ(yτ(k)|x) to β log πθ(yτ(k)|x) + kγ, but the derivation from Eq. (11) is not given. It is unclear whether this modification preserves the Plackett-Luce likelihood interpretation or whether it is simply a heuristic that happens to work. The authors should provide a formal justification or at least an explicit statement that this is a heuristic adaptation, and show that the results are not sensitive to the exact placement of γ (inside vs. outside the log-softmax).","section":"Equation (12)"},{"comment":"The oversampling experiments report SCR improvements for K=15 but the evaluation protocol is not fully specified. When the model outputs 15 joint modes and the metrics are computed for the 'most probable six joint modes', it is unclear whether the top-6 are selected after re-ranking by the scene-level logits under the order-based pairing, or by some other criterion. Since SCR and pSCR depend on which six modes are retained, this choice could affect the reported 64% improvement. The authors should state exactly how the top-6 are chosen in the K=15 experiments.","section":"Section V-C, Figure 3"}],"minor_comments":[{"comment":"In Table I, the first row is labeled 'IntFJMP-Marginal'; this is likely a typo for 'FJMP-Marginal' on the Interaction dataset. Please correct the label for consistency.","section":"Section V-A"},{"comment":"The definition of pSCR uses πθ(yk|x), but it is not explicitly stated that this is the scene-level likelihood obtained by the order-based aggregation of Section IV-C. Please make this dependence explicit at the point of definition.","section":"Equation (14)"},{"comment":"The preference cost Ck in Eq. (9) uses avgFDEk, which requires ground-truth future trajectories. The paper notes that any metric can in principle be used, but it would be helpful to report results with a purely collision-based preference cost (e.g., λ→∞) to demonstrate that the method does not rely on ground-truth knowledge at fine-tuning time.","section":"Section IV-A"},{"comment":"The statement that 'the large decrease in pSCR ... suggests that rearranging agent pairings is the main effect' is important and somewhat undermines the interpretation of SCR improvements as trajectory-shape changes. This should be discussed more prominently, and the paper should distinguish between collision avoidance due to re-pairing versus collision avoidance due to actual trajectory deformation.","section":"Section V-B"}],"recommendation":"major_revision","confidential_remarks":"The paper is well within the scope of the journal and presents a novel idea with thorough experiments. The main risk is the coupling between the training objective and the evaluation metric through the same order-based pairing heuristic. If the authors can provide an evaluation under an independent aggregation (e.g., random pairing or a jointly decoded baseline), the paper would be much stronger. I also encourage them to clarify the formal status of the rank-scaled margin in Eq. (12). I would be willing to re-review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is genuinely the first application of preference optimization to trajectory prediction. The core idea is simple and attractive—fine-tune any likelihood-based predictor with an automatically computed ranking over joint modes, penalizing collisions. The SimPO adaptation to a Plackett-Luce ranking with rank-scaled margins is a reasonable, non-trivial extension, and the experiments span three datasets and four models, with ablations on gamma and lambda. That is real work, and the collision-rate reductions (up to 27% SCR, 57% pSCR for QCNet on AV2) look respectable at first glance.\n\nThe soft spot is the one you flagged: both the training loss and the evaluation metrics (SCR, pSCR) use the same order-based pairing—k-th most likely trajectory per agent forms mode k, with averaged logits. The paper itself says that 'rearranging agent pairings' is the main effect (Sec. V-B). So the model can reduce collisions by permuting the per-agent modes, or by suppressing a single agent's logit inside a colliding mode, without making the joint distribution more coherent in a way that transfers to a planner using a different joint-decoding scheme. The K=15 oversampling experiments are consistent with that mechanism, not evidence against it. This is a genuine confound, not a nitpick.\n\nOther criticisms are more minor: no error bars or significance tests, no code released, and FJMP-Joint on Interaction actually degrades MinJointFDE by 17%, though the mode-collapse explanation is plausible. The preference ranking uses ground-truth FDE, which is fine for a supervised fine-tune, but it does mean the 'any preference metric' claim is untested beyond FDE-plus-collision costs.\n\nOn balance, the paper deserves a serious referee. The originality is clear, the writing is honest, and the limitation statement acknowledges the aggregation dependence. A conditional acceptance should insist on (a) evaluating with an independent aggregation strategy, e.g., sampling joint modes from the model or using a learned pairing; (b) error bars over seeds; and (c) released code. If the gains survive an independent decoder, this is a useful, citable fine-tuning recipe. If not, the paper still stands as a reasonable first step but its central claim needs heavy qualification.","headline":"First preference-optimization paper for trajectory prediction, with a sensible SimPO adaptation and promising collision-rate numbers, but the reported gains are entangled with the order-pairing heuristic used in both training and evaluation.","tokens_in":12604,"tokens_out":3242,"would_cite":true,"duration_ms":36505,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning trajectory predictors on automatically ranked joint futures, using an adapted SimPO preference-optimization loss, substantially reduces scene collision rates while barely hurting accuracy.","keywords":["trajectory prediction","preference optimization","scene consistency","collision rate","SimPO","Plackett-Luce ranking","multi-agent prediction","fine-tuning"],"falsifier":"Take a validation scene where the model's top individual modes cross-collide in every order-based pairing, and compare the probability-weighted collision rate under the paper's pairing with the rate under a randomized or learned pairing of the same agent trajectories; if the improvement disappears or inverts under an alternative pairing, the reported gains are an artifact of the aggregation heuristic rather than of preference optimization.","tokens_in":11658,"feed_emoji":"🚗","tokens_out":5938,"duration_ms":57855,"temperature":0.7,"pith_summary":"The paper asks whether preference optimization, the technique used to align large language models with human preferences, can make multi-agent trajectory predictors internally consistent. It answers yes: by automatically ranking a model's predicted joint futures from best to worst according to a collision-penalizing cost, then fine-tuning with an adapted SimPO loss based on Plackett-Luce ranking, the likelihood of collision-containing modes drops sharply. On Argoverse 2, QCNet's scene collision rate falls 27% and its probability-weighted collision rate falls 57%, while its accuracy metric degrades only 1%. The method adds no inference-time cost and works on both marginal and joint predictors.","feed_headline":"Fine-tuning on ranked futures cuts trajectory collision rates by 57%","feed_subtitle":"Adapted SimPO pushes collision-free joint modes to the top of the ranking, barely hurting accuracy.","key_machinery":"The machinery is an adapted SimPO objective for multi-agent trajectory prediction. Agent-level predictions are paired by likelihood order into $K$ scene modes, and the scene-level log-likelihood is the average of the agents' logits in that mode. The reward becomes $r_\\theta(x,y) = \\beta \\log \\pi_\\theta(y|x)$ with no length normalization, and the loss is the Plackett-Luce ranking loss with a rank-scaled margin $k\\gamma$, so higher-ranked modes must beat lower-ranked modes by an increasing margin. The preference ranking itself comes from an automatic cost combining average per-agent final displacement error with the MotionDiffuser repeller cost, which penalizes pairwise agent distances below a threshold. This loss is applied only on a subset of training scenes that contain collisions or large cost spreads.","core_discovery":"The central claim is that scene consistency in trajectory prediction can be improved by treating joint modes as ranked outputs and optimizing their likelihoods directly, without retraining the architecture or adding inference overhead. The authors adapt SimPO: the reward of a mode is its aggregated log-likelihood $\\beta \\log \\pi_\\theta(y|x)$, the loss uses Plackett-Luce over all $K$ modes, and the target reward margin scales with rank position ($k\\gamma$). Modes are ranked automatically by a cost combining average final displacement error with a repeller collision cost, and collision-free modes are pushed up in probability while collision modes are pushed down. Across Argoverse 2, INTERACTION, and the Waymo Open Motion Dataset, the fine-tuning reduces collision rates for QCNet, FJMP-Marginal, FJMP-Joint, and BeTop while MinJointFDE degrades by only a few percent; oversampling to $K=15$ pushes QCNet's scene collision rate reduction to 64%.","pith_inferences":["The order-based pairing heuristic is the main internal choice; a learned joint scoring head or interaction-aware pairing could make the optimized objective match inference more closely and reduce the small accuracy trade-off.","The method's dependence on mode diversity implies it will work best on anchor- or intention-based decoders; improving mode coverage in weaker decoders may unlock larger consistency gains.","Beyond collisions, the same preference-optimization loop could target rule violations, comfort, or social norms, since any computable ranking can supply the preference signal."],"forward_implications":["Marginal predictors can acquire joint consistency without joint decoding, removing the main reason to switch to heavier joint models.","Planners consuming the fine-tuned predictions see fewer collision modes and much lower probability mass on remaining collision modes, making downstream safety filtering easier.","Increasing the number of sampled modes before fine-tuning amplifies the effect: QCNet with $K=15$ reaches a 64% scene collision rate reduction, showing diversity is a lever for consistency.","Because the ranking metric is pluggable, the same fine-tuning recipe transfers to other definitions of good joint futures with no architectural change."],"supporting_citations":[{"why":"Supplies the base SimPO objective (reference-free reward plus target margin) that the paper adapts to trajectory prediction.","marker":"[8]"},{"why":"Provides the repeller collision cost $R_k$ used in the preference metric to rank modes with collisions below collision-free modes.","marker":"[33]"},{"why":"Defines QCNet, the marginal predictor whose Argoverse 2 fine-tuning yields the headline 27% scene collision rate and 57% probability-weighted collision rate reductions.","marker":"[14]"},{"why":"Defines FJMP, whose marginal and joint variants are fine-tuned on Argoverse 2 and INTERACTION to test the method on both predictor classes.","marker":"[17]"},{"why":"Defines BeTop, the joint model fine-tuned on the Waymo interactive split with 36% scene collision rate and 37% probability-weighted collision rate reductions.","marker":"[21]"},{"why":"Supplies the Argoverse 2 dataset used for the main marginal-model experiments and the interactive-scene sub-split analysis.","marker":"[34]"},{"why":"Supplies the Waymo Open Motion Dataset joint prediction split used to evaluate BeTop.","marker":"[23]"},{"why":"Supplies the INTERACTION dataset used for FJMP experiments and for studying mode collapse.","marker":"[22]"}],"fun_headline_variants":["Ranked future rewards slash trajectory collision rates by 57%","Preference-optimized trajectories: 57% fewer collisions, no extra compute","Fine-tuning on ranked joint modes boosts scene consistency, cuts collisions 57%","SimPO fine-tuning reduces trajectory collisions by 57%, preserving accuracy","Reward-ranked futures cut multi-agent collision rates by 57%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that pairing each agent's k-th most likely trajectory into the k-th scene mode, then averaging the agents' logits, gives a faithful enough joint likelihood; if that pairing is unrepresentative of the true joint distribution, the preference loss optimizes a synthetic objective that may not reflect the collision rates measured at inference.","fun_headline_variants_meta":{"raw":{"variants":["Ranked future rewards slash trajectory collision rates by 57%","Preference-optimized trajectories: 57% fewer collisions, no extra compute","Fine-tuning on ranked joint modes boosts scene consistency, cuts collisions 57%","SimPO fine-tuning reduces trajectory collisions by 57%, preserving accuracy","Reward-ranked futures cut multi-agent collision rates by 57%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000662,"raw_usage":{"total_tokens":3000,"prompt_tokens":898,"completion_tokens":2102,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":514,"completion_tokens_details":{"reasoning_tokens":2007}},"tokens_in":514,"tokens_out":2102,"duration_ms":19136,"temperature":1.0,"reasoning_tokens":2007,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:29:17.103422+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a validation scene where the model's top individual modes cross-collide in every order-based pairing, and compare the probability-weighted collision rate under the paper's pairing with the rate under a randomized or learned pairing of the same agent trajectories; if the improvement disappears or inverts under an alternative pairing, the reported gains are an artifact of the aggregation heuristic rather than of preference optimization.","supporting_citations":[{"cited_title":"SimPO: Simple preference optimization with a reference-free reward,","cited_arxiv_id":null,"evidence_quote":"Supplies the base SimPO objective (reference-free reward plus target margin) that the paper adapts to trajectory prediction."},{"cited_title":"MotionDiffuser: Controllable multi-agent motion prediction using dif- fusion,","cited_arxiv_id":null,"evidence_quote":"Provides the repeller collision cost $R_k$ used in the preference metric to rank modes with collisions below collision-free modes."},{"cited_title":"FJMP: Factorized joint multi-agent motion prediction over learned directed acyclic interac- tion graphs,","cited_arxiv_id":null,"evidence_quote":"Defines FJMP, whose marginal and joint variants are fine-tuned on Argoverse 2 and INTERACTION to test the method on both predictor classes."},{"cited_title":"Reasoning multi-agent behavioral topology for interactive autonomous driving,","cited_arxiv_id":null,"evidence_quote":"Defines BeTop, the joint model fine-tuned on the Waymo interactive split with 36% scene collision rate and 37% probability-weighted collision rate reductions."},{"cited_title":"Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting,","cited_arxiv_id":null,"evidence_quote":"Supplies the Argoverse 2 dataset used for the main marginal-model experiments and the interactive-scene sub-split analysis."},{"cited_title":"Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset,","cited_arxiv_id":null,"evidence_quote":"Supplies the Waymo Open Motion Dataset joint prediction split used to evaluate BeTop."}],"review_version":1}